{"id":9239,"url":"https://patchwork.libcamera.org/api/1.1/patches/9239/?format=json","web_url":"https://patchwork.libcamera.org/patch/9239/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200805151507.227503-13-kieran.bingham@ideasonboard.com>","date":"2020-08-05T15:15:06","name":"[libcamera-devel,v4,12/13] android: Introduce JPEG compression","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"4b622e2d2001e5149d014308ad7c2e56b95466c8","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/1.1/people/4/?format=json","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/9239/mbox/","series":[{"id":1198,"url":"https://patchwork.libcamera.org/api/1.1/series/1198/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1198","date":"2020-08-05T15:14:54","name":"android: JPEG support","version":4,"mbox":"https://patchwork.libcamera.org/series/1198/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/9239/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/9239/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 15B89BD87B\n\tfor ;\n\tWed, 5 Aug 2020 15:15:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CBF71608E8;\n\tWed, 5 Aug 2020 17:15:20 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 602FC60392\n\tfor ;\n\tWed, 5 Aug 2020 17:15:17 +0200 (CEST)","from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net\n\t[86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D87B011F6;\n\tWed, 5 Aug 2020 17:15:16 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Q6Z5t7IK\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1596640517;\n\tbh=7cq7S51KXFxlSSgAB/IfwBLRFxPEnXdsH9LEfOb5HxA=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=Q6Z5t7IKTL87z9ATVFEXqYLxEYaf/+tzI8priaNdSv3nHWv1yAADxTSLGgCZfwi+R\n\t6nkwwO2jr4ZGWjnGzfzSX/Lbu7QBefNPC0axdCbBerITSwzeg8yIRW+Z1fM4aIzoVa\n\tGtKsc8b1PuoQttvY46qIbX/LfrWo2nQWzkeF2/dg=","From":"Kieran Bingham ","To":"libcamera devel ","Date":"Wed, 5 Aug 2020 16:15:06 +0100","Message-Id":"<20200805151507.227503-13-kieran.bingham@ideasonboard.com>","X-Mailer":"git-send-email 2.25.1","In-Reply-To":"<20200805151507.227503-1-kieran.bingham@ideasonboard.com>","References":"<20200805151507.227503-1-kieran.bingham@ideasonboard.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v4 12/13] android: Introduce JPEG\n\tcompression","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Provide a compressor interface and implement a JPEG compressor using\nlibjpeg.\n\nReviewed-by: Jacopo Mondi \nSigned-off-by: Kieran Bingham \n---\n\nv2:\n\n - Convert to use the libcamera format information rather than duplicating it ourselves.\n - Not easy to get the horizontal subsampling\n - not easy to determine if we have an nvSwap...\n\nv3:\n - Fix frame.error return value (negative inversion)\n - Fix comments\n - constify data table\n - set jpeg image lenght to zero after free\n - set override on class interface functions.\n - Remove ununsed fucntion prototype\n - Rename pixelFormatInfo to pixelFormatInfo_\n\nv4:\n - Add todo regarding strides from buffers\n - Rename to compressor_libjpeg rather than compressor_jpeg\n - Define android dependencies in their own variable in meson\n---\n src/android/jpeg/compressor.h | 25 +++\n src/android/jpeg/compressor_libjpeg.cpp | 217 ++++++++++++++++++++++++\n src/android/jpeg/compressor_libjpeg.h | 41 +++++\n src/android/meson.build | 5 +\n src/libcamera/meson.build | 2 +\n 5 files changed, 290 insertions(+)\n create mode 100644 src/android/jpeg/compressor.h\n create mode 100644 src/android/jpeg/compressor_libjpeg.cpp\n create mode 100644 src/android/jpeg/compressor_libjpeg.h","diff":"diff --git a/src/android/jpeg/compressor.h b/src/android/jpeg/compressor.h\nnew file mode 100644\nindex 000000000000..113693b8cdb6\n--- /dev/null\n+++ b/src/android/jpeg/compressor.h\n@@ -0,0 +1,25 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * compressor.h - Image compression interface\n+ */\n+#ifndef __ANDROID_JPEG_COMPRESSOR_H__\n+#define __ANDROID_JPEG_COMPRESSOR_H__\n+\n+#include \n+#include \n+#include \n+\n+class Compressor\n+{\n+public:\n+\tusing CompressedImage = libcamera::Span;\n+\n+\tvirtual ~Compressor() {};\n+\n+\tvirtual int configure(const libcamera::StreamConfiguration &cfg) = 0;\n+\tvirtual int compress(const libcamera::FrameBuffer *source, const CompressedImage &image) = 0;\n+};\n+\n+#endif /* __ANDROID_JPEG_COMPRESSOR_H__ */\ndiff --git a/src/android/jpeg/compressor_libjpeg.cpp b/src/android/jpeg/compressor_libjpeg.cpp\nnew file mode 100644\nindex 000000000000..5c843ab1dd47\n--- /dev/null\n+++ b/src/android/jpeg/compressor_libjpeg.cpp\n@@ -0,0 +1,217 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * compressor_libjpeg.cpp - JPEG compression using libjpeg native API\n+ */\n+\n+#include \"compressor_libjpeg.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/log.h\"\n+\n+using namespace libcamera;\n+\n+LOG_DEFINE_CATEGORY(JPEG)\n+\n+namespace {\n+\n+struct JPEGPixelFormatInfo {\n+\tJ_COLOR_SPACE colorSpace;\n+\tconst PixelFormatInfo &pixelFormatInfo;\n+\tbool nvSwap;\n+};\n+\n+const std::map pixelInfo{\n+\t{ formats::R8, { JCS_GRAYSCALE, PixelFormatInfo::info(formats::R8), false } },\n+\n+\t{ formats::RGB888, { JCS_EXT_BGR, PixelFormatInfo::info(formats::RGB888), false } },\n+\t{ formats::BGR888, { JCS_EXT_RGB, PixelFormatInfo::info(formats::BGR888), false } },\n+\n+\t{ formats::NV12, { JCS_YCbCr, PixelFormatInfo::info(formats::NV12), false } },\n+\t{ formats::NV21, { JCS_YCbCr, PixelFormatInfo::info(formats::NV21), true } },\n+\t{ formats::NV16, { JCS_YCbCr, PixelFormatInfo::info(formats::NV16), false } },\n+\t{ formats::NV61, { JCS_YCbCr, PixelFormatInfo::info(formats::NV61), true } },\n+\t{ formats::NV24, { JCS_YCbCr, PixelFormatInfo::info(formats::NV24), false } },\n+\t{ formats::NV42, { JCS_YCbCr, PixelFormatInfo::info(formats::NV42), true } },\n+};\n+\n+const struct JPEGPixelFormatInfo &findPixelInfo(const PixelFormat &format)\n+{\n+\tstatic const struct JPEGPixelFormatInfo invalidPixelFormat {\n+\t\tJCS_UNKNOWN, PixelFormatInfo(), false\n+\t};\n+\n+\tconst auto iter = pixelInfo.find(format);\n+\tif (iter == pixelInfo.end()) {\n+\t\tLOG(JPEG, Error) << \"Unsupported pixel format for JPEG compressor: \"\n+\t\t\t\t << format.toString();\n+\t\treturn invalidPixelFormat;\n+\t}\n+\n+\treturn iter->second;\n+}\n+\n+} /* namespace */\n+\n+CompressorLibJpeg::CompressorLibJpeg()\n+\t: quality_(95)\n+{\n+\t/* \\todo Expand error handling coverage with a custom handler. */\n+\tcompress_.err = jpeg_std_error(&jerr_);\n+\n+\tjpeg_create_compress(&compress_);\n+}\n+\n+CompressorLibJpeg::~CompressorLibJpeg()\n+{\n+\tjpeg_destroy_compress(&compress_);\n+}\n+\n+int CompressorLibJpeg::configure(const StreamConfiguration &cfg)\n+{\n+\tconst struct JPEGPixelFormatInfo info = findPixelInfo(cfg.pixelFormat);\n+\tif (info.colorSpace == JCS_UNKNOWN)\n+\t\treturn -ENOTSUP;\n+\n+\tcompress_.image_width = cfg.size.width;\n+\tcompress_.image_height = cfg.size.height;\n+\tcompress_.in_color_space = info.colorSpace;\n+\n+\tcompress_.input_components = info.colorSpace == JCS_GRAYSCALE ? 1 : 3;\n+\n+\tjpeg_set_defaults(&compress_);\n+\tjpeg_set_quality(&compress_, quality_, TRUE);\n+\n+\tpixelFormatInfo_ = &info.pixelFormatInfo;\n+\n+\tnv_ = pixelFormatInfo_->numPlanes() == 2;\n+\tnvSwap_ = info.nvSwap;\n+\n+\treturn 0;\n+}\n+\n+void CompressorLibJpeg::compressRGB(const libcamera::MappedBuffer *frame)\n+{\n+\tunsigned char *src = static_cast(frame->maps()[0].data());\n+\t/* \\todo Stride information should come from buffer configuration. */\n+\tunsigned int stride = pixelFormatInfo_->stride(compress_.image_width, 0);\n+\n+\tJSAMPROW row_pointer[1];\n+\n+\twhile (compress_.next_scanline < compress_.image_height) {\n+\t\trow_pointer[0] = &src[compress_.next_scanline * stride];\n+\t\tjpeg_write_scanlines(&compress_, row_pointer, 1);\n+\t}\n+}\n+\n+/*\n+ * Compress the incoming buffer from a supported NV format.\n+ * This naively unpacks the semi-planar NV12 to a YUV888 format for libjpeg.\n+ */\n+void CompressorLibJpeg::compressNV(const libcamera::MappedBuffer *frame)\n+{\n+\tuint8_t tmprowbuf[compress_.image_width * 3];\n+\n+\t/*\n+\t * \\todo Use the raw api, and only unpack the cb/cr samples to new line\n+\t * buffers. If possible, see if we can set appropriate pixel strides\n+\t * too to save even that copy.\n+\t *\n+\t * Possible hints at:\n+\t * https://sourceforge.net/p/libjpeg/mailman/message/30815123/\n+\t */\n+\tunsigned int y_stride = pixelFormatInfo_->stride(compress_.image_width, 0);\n+\tunsigned int c_stride = pixelFormatInfo_->stride(compress_.image_width, 1);\n+\n+\tunsigned int horzSubSample = 2 * compress_.image_width / c_stride;\n+\tunsigned int vertSubSample = pixelFormatInfo_->planes[1].verticalSubSampling;\n+\n+\tunsigned int c_inc = horzSubSample == 1 ? 2 : 0;\n+\tunsigned int cb_pos = nvSwap_ ? 1 : 0;\n+\tunsigned int cr_pos = nvSwap_ ? 0 : 1;\n+\n+\tconst unsigned char *src = static_cast(frame->maps()[0].data());\n+\tconst unsigned char *src_c = src + y_stride * compress_.image_height;\n+\n+\tJSAMPROW row_pointer[1];\n+\trow_pointer[0] = &tmprowbuf[0];\n+\n+\tfor (unsigned int y = 0; y < compress_.image_height; y++) {\n+\t\tunsigned char *dst = &tmprowbuf[0];\n+\n+\t\tconst unsigned char *src_y = src + y * compress_.image_width;\n+\t\tconst unsigned char *src_cb = src_c + (y / vertSubSample) * c_stride + cb_pos;\n+\t\tconst unsigned char *src_cr = src_c + (y / vertSubSample) * c_stride + cr_pos;\n+\n+\t\tfor (unsigned int x = 0; x < compress_.image_width; x += 2) {\n+\t\t\tdst[0] = *src_y;\n+\t\t\tdst[1] = *src_cb;\n+\t\t\tdst[2] = *src_cr;\n+\t\t\tsrc_y++;\n+\t\t\tsrc_cb += c_inc;\n+\t\t\tsrc_cr += c_inc;\n+\t\t\tdst += 3;\n+\n+\t\t\tdst[0] = *src_y;\n+\t\t\tdst[1] = *src_cb;\n+\t\t\tdst[2] = *src_cr;\n+\t\t\tsrc_y++;\n+\t\t\tsrc_cb += 2;\n+\t\t\tsrc_cr += 2;\n+\t\t\tdst += 3;\n+\t\t}\n+\n+\t\tjpeg_write_scanlines(&compress_, row_pointer, 1);\n+\t}\n+}\n+\n+int CompressorLibJpeg::compress(const FrameBuffer *source, const CompressedImage &dest)\n+{\n+\tMappedFrameBuffer frame(source, PROT_READ);\n+\tif (!frame.isValid()) {\n+\t\tLOG(JPEG, Error) << \"Failed to map FrameBuffer : \"\n+\t\t\t\t << strerror(frame.error());\n+\t\treturn frame.error();\n+\t}\n+\n+\tunsigned char *destination = dest.data();\n+\tunsigned long size = dest.size();\n+\n+\t/*\n+\t * The jpeg_mem_dest will reallocate if the required size is not\n+\t * sufficient. That means the output won't be written to the correct\n+\t * buffers.\n+\t *\n+\t * \\todo Implement our own custom memory destination to prevent\n+\t * reallocation and prefer failure with correct reporting.\n+\t */\n+\tjpeg_mem_dest(&compress_, &destination, &size);\n+\n+\tjpeg_start_compress(&compress_, TRUE);\n+\n+\tLOG(JPEG, Debug) << \"JPEG Compress Starting:\" << compress_.image_width\n+\t\t\t << \"x\" << compress_.image_height;\n+\n+\tif (nv_)\n+\t\tcompressNV(&frame);\n+\telse\n+\t\tcompressRGB(&frame);\n+\n+\tjpeg_finish_compress(&compress_);\n+\n+\treturn size;\n+}\ndiff --git a/src/android/jpeg/compressor_libjpeg.h b/src/android/jpeg/compressor_libjpeg.h\nnew file mode 100644\nindex 000000000000..ba4650832332\n--- /dev/null\n+++ b/src/android/jpeg/compressor_libjpeg.h\n@@ -0,0 +1,41 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * compressor_libjpeg.h - JPEG compression using libjpeg\n+ */\n+#ifndef __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__\n+#define __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__\n+\n+#include \"compressor.h\"\n+\n+#include \"libcamera/internal/buffer.h\"\n+#include \"libcamera/internal/formats.h\"\n+\n+#include \n+\n+class CompressorLibJpeg : public Compressor\n+{\n+public:\n+\tCompressorLibJpeg();\n+\t~CompressorLibJpeg();\n+\n+\tint configure(const libcamera::StreamConfiguration &cfg) override;\n+\tint compress(const libcamera::FrameBuffer *source, const CompressedImage &jpeg) override;\n+\n+private:\n+\tvoid compressRGB(const libcamera::MappedBuffer *frame);\n+\tvoid compressNV(const libcamera::MappedBuffer *frame);\n+\n+\tstruct jpeg_compress_struct compress_;\n+\tstruct jpeg_error_mgr jerr_;\n+\n+\tunsigned int quality_;\n+\n+\tconst libcamera::PixelFormatInfo *pixelFormatInfo_;\n+\n+\tbool nv_;\n+\tbool nvSwap_;\n+};\n+\n+#endif /* __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__ */\ndiff --git a/src/android/meson.build b/src/android/meson.build\nindex 822cad621f01..7dd375e73eff 100644\n--- a/src/android/meson.build\n+++ b/src/android/meson.build\n@@ -6,12 +6,17 @@ android_hal_sources = files([\n 'camera_device.cpp',\n 'camera_metadata.cpp',\n 'camera_ops.cpp',\n+ 'jpeg/compressor_libjpeg.cpp',\n ])\n \n android_camera_metadata_sources = files([\n 'metadata/camera_metadata.c',\n ])\n \n+android_deps = [\n+ dependency('libjpeg'),\n+]\n+\n android_camera_metadata = static_library('camera_metadata',\n android_camera_metadata_sources,\n include_directories : android_includes)\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex 3aad4386ffc2..8ad0a1b0740f 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -124,6 +124,8 @@ if get_option('android')\n libcamera_sources += android_hal_sources\n includes += android_includes\n libcamera_link_with += android_camera_metadata\n+\n+ libcamera_deps += android_deps\n endif\n \n # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag.\n","prefixes":["libcamera-devel","v4","12/13"]}