From patchwork Wed Apr 6 09:41:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 15628 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7EA4FC3260 for ; Wed, 6 Apr 2022 10:34:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EC52B604B8; Wed, 6 Apr 2022 12:34:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649241268; bh=79pZ+B1sOBWjlpNvWpS0qZfeCRaBeErcY1fLaUyxn4I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ynmd///U2aK+uy2BPqsE1osG3UkTbUMW8Ly7+WicJtOQYCbSRMYNpOGjZYHSpUvrB 2SYWw7PNAg5Ig4mqJBcnClVh5W7SwjQiC1tBz02vMx9vYQakTPAB0QoG+/Gyoa+c/0 tAbSDg1Nyt7U56QE13veDDw3LK6xhbdABuQC8P8Wma2yRMdWsSRltytEqPVReAZ85L xGoBxjMlNT3GS7AO+zEQ2Nqb34C/aKt/xIwXlNfLpvIWd3Fysh9FFMx2PIM6YAlgK0 2rI1LpibhRUWjlI2QIQSQfMXOt1Lpeh8clhRG2pFFGcifb7KP8rDl3IfLQHCqkOq8g OqVjU58RlK7ow== Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 117A46563F for ; Wed, 6 Apr 2022 11:41:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cJmVaYRy"; dkim-atps=neutral Received: by mail-pj1-x1033.google.com with SMTP id nt14-20020a17090b248e00b001ca601046a4so5310395pjb.0 for ; Wed, 06 Apr 2022 02:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+PkXHup3JeH0yN48Y3QrIgJ80QxoxICNR+2RvPT0ZcQ=; b=cJmVaYRyI8JMETZlCT5z9KWBkUY2AL16Ti7Q8jm3o7gImTw52B4oTTp2SQt8icKPwp 5R9kWRVxrnjF0nGDGSrhS/gXeU2/CiTQVajMAPCZT93RyTLyM0cSaMMD+ZOu2O5srXgR uUDBpsOGnw12DcCHU8cK8R43mZIYb5TAa+ea4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+PkXHup3JeH0yN48Y3QrIgJ80QxoxICNR+2RvPT0ZcQ=; b=UTLP6qbdHfhVbr6Gax59QQcQZmOB8S9j7UTrI+4hjdPUscHBJvmDqYAlBUSihfVz5i Ly4Pdmt4aVUwTsOzRfgPGU3SmhGp1wyCs9ScPea0paYE+B5KQN/c86SmCtggHkS1VkQZ 3nWRBLez7BOwx+0Clg7kFbhphQMTOcNPFsV4NvUDdG1+2V7KEsDUNfruS1yhckcxPrmf 5fyxOi2ETLDRsEvxAruGrDdeo1D1obATGqsGfYr63k0Q0DPYIrtXcfpOXeMG5BIxedIZ pUjBeYpnJ+DyjuP/gkTRJpU+PWGUrRKFIACgy1pbD50RzlzncailL8gPTMh8BHn9APPW XwAA== X-Gm-Message-State: AOAM533rBwfxq7Et5GapT3y7p1dKrukMdeMU1Qcz2rfa7P1Al2cUuxII oTyzhGtFgFCwtelPXbjk9WEHOTEfJSycLQ== X-Google-Smtp-Source: ABdhPJwbSh7Sb2G5uI0uuf4f7z3OSs+0ocVJiFJ1XmX8feETRKuVTIHbDQdDWK5Qoxd3Jy39vDa9KQ== X-Received: by 2002:a17:902:b213:b0:156:5a00:325f with SMTP id t19-20020a170902b21300b001565a00325fmr7572649plr.127.1649238108700; Wed, 06 Apr 2022 02:41:48 -0700 (PDT) Received: from chenghaoyang-p920.tpe.corp.google.com ([2401:fa00:1:17:42f3:c4c9:1892:e888]) by smtp.gmail.com with ESMTPSA id mq6-20020a17090b380600b001c6357f146csm6071855pjb.12.2022.04.06.02.41.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Apr 2022 02:41:48 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 17:41:30 +0800 Message-Id: <20220406094130.189862-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog In-Reply-To: <20220406094130.189862-1-chenghaoyang@chromium.org> References: <20220406094130.189862-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 06 Apr 2022 12:34:26 +0200 Subject: [libcamera-devel] [PATCH 1/1] Add CrOS JEA implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This CL uses CrOS JpegCompressor with potential HW accelerator to do JPEG encoding. As CrOS JpegCompressor might need file descriptors to get the source data and pass the jpeg result, this CL extends FrameBuffer in the android source code as Android_FrameBuffer, which stores the buffer_handle_t when constructing the frame buffer, and adds a getter function to access it. This CL also redefines src/android/jpeg/encoder interfaces and adds Encoder::generateThumbnail, which might also be accelerated by CrOS HW. It simplifies PostProcessorJpeg's logic when generating the thumbnail. The original implementation is then moved into the EncoderLibJpeg::generateThumbnail. --- include/libcamera/framebuffer.h | 3 +- src/android/android_framebuffer.cpp | 32 ++++++++ src/android/android_framebuffer.h | 28 +++++++ src/android/camera_device.cpp | 3 +- src/android/cros/camera3_hal.cpp | 3 + src/android/frame_buffer_allocator.h | 37 +++++---- src/android/jpeg/cros_post_processor_jpeg.cpp | 14 ++++ src/android/jpeg/encoder.h | 9 +- src/android/jpeg/encoder_jea.cpp | 82 +++++++++++++++++++ src/android/jpeg/encoder_jea.h | 35 ++++++++ src/android/jpeg/encoder_libjpeg.cpp | 70 ++++++++++++++++ src/android/jpeg/encoder_libjpeg.h | 21 ++++- .../jpeg/generic_post_processor_jpeg.cpp | 14 ++++ src/android/jpeg/meson.build | 16 ++++ src/android/jpeg/post_processor_jpeg.cpp | 60 ++------------ src/android/jpeg/post_processor_jpeg.h | 11 +-- src/android/meson.build | 6 +- .../mm/cros_frame_buffer_allocator.cpp | 13 +-- .../mm/generic_frame_buffer_allocator.cpp | 11 +-- 19 files changed, 367 insertions(+), 101 deletions(-) create mode 100644 src/android/android_framebuffer.cpp create mode 100644 src/android/android_framebuffer.h create mode 100644 src/android/jpeg/cros_post_processor_jpeg.cpp create mode 100644 src/android/jpeg/encoder_jea.cpp create mode 100644 src/android/jpeg/encoder_jea.h create mode 100644 src/android/jpeg/generic_post_processor_jpeg.cpp create mode 100644 src/android/jpeg/meson.build diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index de172d97..c902cc18 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -46,7 +46,7 @@ private: std::vector planes_; }; -class FrameBuffer final : public Extensible +class FrameBuffer : public Extensible { LIBCAMERA_DECLARE_PRIVATE() @@ -61,6 +61,7 @@ public: FrameBuffer(const std::vector &planes, unsigned int cookie = 0); FrameBuffer(std::unique_ptr d, const std::vector &planes, unsigned int cookie = 0); + virtual ~FrameBuffer() {} const std::vector &planes() const { return planes_; } Request *request() const; diff --git a/src/android/android_framebuffer.cpp b/src/android/android_framebuffer.cpp new file mode 100644 index 00000000..1ff7018e --- /dev/null +++ b/src/android/android_framebuffer.cpp @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * android_framebuffer.cpp - Android Frame buffer handling + */ + +#include "android_framebuffer.h" + +#include + +AndroidFrameBuffer::AndroidFrameBuffer( + buffer_handle_t handle, + std::unique_ptr d, + const std::vector &planes, + unsigned int cookie) + : FrameBuffer(std::move(d), planes, cookie), handle_(handle) +{ +} + +AndroidFrameBuffer::AndroidFrameBuffer( + buffer_handle_t handle, + const std::vector &planes, + unsigned int cookie) + : FrameBuffer(planes, cookie), handle_(handle) +{ +} + +buffer_handle_t AndroidFrameBuffer::getHandle() const +{ + return handle_; +} diff --git a/src/android/android_framebuffer.h b/src/android/android_framebuffer.h new file mode 100644 index 00000000..49df9756 --- /dev/null +++ b/src/android/android_framebuffer.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * android_framebuffer.h - Android Frame buffer handling + */ + +#pragma once + +#include "libcamera/internal/framebuffer.h" + +#include + +class AndroidFrameBuffer final : public libcamera::FrameBuffer +{ +public: + AndroidFrameBuffer( + buffer_handle_t handle, std::unique_ptr d, + const std::vector &planes, + unsigned int cookie = 0); + AndroidFrameBuffer(buffer_handle_t handle, + const std::vector &planes, + unsigned int cookie = 0); + buffer_handle_t getHandle() const; + +private: + buffer_handle_t handle_ = nullptr; +}; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..643b4dee 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -25,6 +25,7 @@ #include "system/graphics.h" +#include "android_framebuffer.h" #include "camera_buffer.h" #include "camera_hal_config.h" #include "camera_ops.h" @@ -754,7 +755,7 @@ CameraDevice::createFrameBuffer(const buffer_handle_t camera3buffer, planes[i].length = buf.size(i); } - return std::make_unique(planes); + return std::make_unique(camera3buffer, planes); } int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) diff --git a/src/android/cros/camera3_hal.cpp b/src/android/cros/camera3_hal.cpp index fb863b5f..ea5577f0 100644 --- a/src/android/cros/camera3_hal.cpp +++ b/src/android/cros/camera3_hal.cpp @@ -9,8 +9,11 @@ #include "../camera_hal_manager.h" +cros::CameraMojoChannelManagerToken *g_cros_camera_token = nullptr; + static void set_up([[maybe_unused]] cros::CameraMojoChannelManagerToken *token) { + g_cros_camera_token = token; } static void tear_down() diff --git a/src/android/frame_buffer_allocator.h b/src/android/frame_buffer_allocator.h index 5d2eeda1..e26422a3 100644 --- a/src/android/frame_buffer_allocator.h +++ b/src/android/frame_buffer_allocator.h @@ -13,9 +13,10 @@ #include #include -#include #include +#include "android_framebuffer.h" + class CameraDevice; class PlatformFrameBufferAllocator : libcamera::Extensible @@ -31,25 +32,25 @@ public: * Note: The returned FrameBuffer needs to be destroyed before * PlatformFrameBufferAllocator is destroyed. */ - std::unique_ptr allocate( + std::unique_ptr allocate( int halPixelFormat, const libcamera::Size &size, uint32_t usage); }; -#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION \ -PlatformFrameBufferAllocator::PlatformFrameBufferAllocator( \ - CameraDevice *const cameraDevice) \ - : Extensible(std::make_unique(cameraDevice)) \ -{ \ -} \ -PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator() \ -{ \ -} \ -std::unique_ptr \ -PlatformFrameBufferAllocator::allocate(int halPixelFormat, \ - const libcamera::Size &size, \ - uint32_t usage) \ -{ \ - return _d()->allocate(halPixelFormat, size, usage); \ -} +#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION \ + PlatformFrameBufferAllocator::PlatformFrameBufferAllocator( \ + CameraDevice *const cameraDevice) \ + : Extensible(std::make_unique(cameraDevice)) \ + { \ + } \ + PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator() \ + { \ + } \ + std::unique_ptr \ + PlatformFrameBufferAllocator::allocate(int halPixelFormat, \ + const libcamera::Size &size, \ + uint32_t usage) \ + { \ + return _d()->allocate(halPixelFormat, size, usage); \ + } #endif /* __ANDROID_FRAME_BUFFER_ALLOCATOR_H__ */ diff --git a/src/android/jpeg/cros_post_processor_jpeg.cpp b/src/android/jpeg/cros_post_processor_jpeg.cpp new file mode 100644 index 00000000..7020f0d0 --- /dev/null +++ b/src/android/jpeg/cros_post_processor_jpeg.cpp @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * cros_post_processor_jpeg.cpp - CrOS JPEG Post Processor + */ + +#include "encoder_jea.h" +#include "post_processor_jpeg.h" + +void PostProcessorJpeg::SetEncoder() +{ + encoder_ = std::make_unique(); +} diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h index b974d367..6d527d91 100644 --- a/src/android/jpeg/encoder.h +++ b/src/android/jpeg/encoder.h @@ -12,14 +12,19 @@ #include #include +#include "../camera_request.h" + class Encoder { public: virtual ~Encoder() = default; virtual int configure(const libcamera::StreamConfiguration &cfg) = 0; - virtual int encode(const libcamera::FrameBuffer &source, - libcamera::Span destination, + virtual int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer, libcamera::Span exifData, unsigned int quality) = 0; + virtual int generateThumbnail(const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, + unsigned int quality, + std::vector *thumbnail) = 0; }; diff --git a/src/android/jpeg/encoder_jea.cpp b/src/android/jpeg/encoder_jea.cpp new file mode 100644 index 00000000..838e8647 --- /dev/null +++ b/src/android/jpeg/encoder_jea.cpp @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * encoder_jea.cpp - JPEG encoding using CrOS JEA + */ + +#include "encoder_jea.h" + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include + +#include "../android_framebuffer.h" + +extern cros::CameraMojoChannelManagerToken *g_cros_camera_token; + +EncoderJea::EncoderJea() = default; + +EncoderJea::~EncoderJea() = default; + +int EncoderJea::configure(const libcamera::StreamConfiguration &cfg) +{ + size_ = cfg.size; + + if (jpeg_compressor_.get()) + return 0; + + if (g_cros_camera_token == nullptr) + return -ENOTSUP; + + jpeg_compressor_ = cros::JpegCompressor::GetInstance(g_cros_camera_token); + + return 0; +} + +int EncoderJea::encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer, + libcamera::Span exifData, + unsigned int quality) +{ + if (!jpeg_compressor_.get()) + return -1; + + uint32_t out_data_size = 0; + + if (!jpeg_compressor_->CompressImageFromHandle( + dynamic_cast( + streamBuffer->srcBuffer) + ->getHandle(), + *streamBuffer->camera3Buffer, size_.width, size_.height, quality, + exifData.data(), exifData.size(), &out_data_size)) { + return -1; + } + + return out_data_size; +} + +int EncoderJea::generateThumbnail(const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, + unsigned int quality, + std::vector *thumbnail) +{ + if (!jpeg_compressor_.get()) + return -1; + + libcamera::MappedFrameBuffer frame(&source, libcamera::MappedFrameBuffer::MapFlag::Read); + + if (frame.planes().empty()) + return 0; + + uint32_t out_data_size = 0; + + if (!jpeg_compressor_->GenerateThumbnail(frame.planes()[0].data(), + size_.width, size_.height, targetSize.width, targetSize.height, + quality, thumbnail->size(), thumbnail->data(), &out_data_size)) { + return -1; + } + + return out_data_size; +} diff --git a/src/android/jpeg/encoder_jea.h b/src/android/jpeg/encoder_jea.h new file mode 100644 index 00000000..d5c9f1f7 --- /dev/null +++ b/src/android/jpeg/encoder_jea.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * encoder_jea.h - JPEG encoding using CrOS JEA + */ + +#pragma once + +#include + +#include + +#include "encoder.h" + +class EncoderJea : public Encoder +{ +public: + EncoderJea(); + ~EncoderJea(); + + int configure(const libcamera::StreamConfiguration &cfg) override; + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer, + libcamera::Span exifData, + unsigned int quality) override; + int generateThumbnail(const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, + unsigned int quality, + std::vector *thumbnail) override; + +private: + libcamera::Size size_; + + std::unique_ptr jpeg_compressor_; +}; diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp index 21a3b33d..b5591e33 100644 --- a/src/android/jpeg/encoder_libjpeg.cpp +++ b/src/android/jpeg/encoder_libjpeg.cpp @@ -24,6 +24,8 @@ #include "libcamera/internal/formats.h" #include "libcamera/internal/mapped_framebuffer.h" +#include "../camera_buffer.h" + using namespace libcamera; LOG_DECLARE_CATEGORY(JPEG) @@ -82,8 +84,17 @@ EncoderLibJpeg::~EncoderLibJpeg() } int EncoderLibJpeg::configure(const StreamConfiguration &cfg) +{ + thumbnailer_.configure(cfg.size, cfg.pixelFormat); + cfg_ = cfg; + + return internalConfigure(cfg); +} + +int EncoderLibJpeg::internalConfigure(const StreamConfiguration &cfg) { const struct JPEGPixelFormatInfo info = findPixelInfo(cfg.pixelFormat); + if (info.colorSpace == JCS_UNKNOWN) return -ENOTSUP; @@ -178,6 +189,65 @@ void EncoderLibJpeg::compressNV(const std::vector> &planes) } } +int EncoderLibJpeg::encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer, + libcamera::Span exifData, + unsigned int quality) +{ + internalConfigure(cfg_); + return encode(*streamBuffer->srcBuffer, streamBuffer->dstBuffer.get()->plane(0), exifData, quality); +} + +int EncoderLibJpeg::generateThumbnail( + const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, + unsigned int quality, + std::vector *thumbnail) +{ + /* Stores the raw scaled-down thumbnail bytes. */ + std::vector rawThumbnail; + + thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail); + + StreamConfiguration thCfg; + thCfg.size = targetSize; + thCfg.pixelFormat = thumbnailer_.pixelFormat(); + int ret = internalConfigure(thCfg); + + if (!rawThumbnail.empty() && !ret) { + /* + * \todo Avoid value-initialization of all elements of the + * vector. + */ + thumbnail->resize(rawThumbnail.size()); + + /* + * Split planes manually as the encoder expects a vector of + * planes. + * + * \todo Pass a vector of planes directly to + * Thumbnailer::createThumbnailer above and remove the manual + * planes split from here. + */ + std::vector> thumbnailPlanes; + const PixelFormatInfo &formatNV12 = PixelFormatInfo::info(formats::NV12); + size_t yPlaneSize = formatNV12.planeSize(targetSize, 0); + size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1); + thumbnailPlanes.push_back({ rawThumbnail.data(), yPlaneSize }); + thumbnailPlanes.push_back({ rawThumbnail.data() + yPlaneSize, uvPlaneSize }); + + int jpeg_size = encode(thumbnailPlanes, *thumbnail, {}, quality); + thumbnail->resize(jpeg_size); + + LOG(JPEG, Debug) + << "Thumbnail compress returned " + << jpeg_size << " bytes"; + + return jpeg_size; + } + + return -1; +} + int EncoderLibJpeg::encode(const FrameBuffer &source, Span dest, Span exifData, unsigned int quality) { diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h index 1b3ac067..56b27bae 100644 --- a/src/android/jpeg/encoder_libjpeg.h +++ b/src/android/jpeg/encoder_libjpeg.h @@ -15,6 +15,8 @@ #include +#include "thumbnailer.h" + class EncoderLibJpeg : public Encoder { public: @@ -22,19 +24,32 @@ public: ~EncoderLibJpeg(); int configure(const libcamera::StreamConfiguration &cfg) override; + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer, + libcamera::Span exifData, + unsigned int quality) override; + int generateThumbnail( + const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, + unsigned int quality, + std::vector *thumbnail) override; + +private: + int internalConfigure(const libcamera::StreamConfiguration &cfg); + int encode(const libcamera::FrameBuffer &source, libcamera::Span destination, libcamera::Span exifData, - unsigned int quality) override; + unsigned int quality); int encode(const std::vector> &planes, libcamera::Span destination, libcamera::Span exifData, unsigned int quality); -private: void compressRGB(const std::vector> &planes); void compressNV(const std::vector> &planes); + libcamera::StreamConfiguration cfg_; + struct jpeg_compress_struct compress_; struct jpeg_error_mgr jerr_; @@ -42,4 +57,6 @@ private: bool nv_; bool nvSwap_; + + Thumbnailer thumbnailer_; }; diff --git a/src/android/jpeg/generic_post_processor_jpeg.cpp b/src/android/jpeg/generic_post_processor_jpeg.cpp new file mode 100644 index 00000000..890f6972 --- /dev/null +++ b/src/android/jpeg/generic_post_processor_jpeg.cpp @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * generic_post_processor_jpeg.cpp - Generic JPEG Post Processor + */ + +#include "encoder_libjpeg.h" +#include "post_processor_jpeg.h" + +void PostProcessorJpeg::SetEncoder() +{ + encoder_ = std::make_unique(); +} diff --git a/src/android/jpeg/meson.build b/src/android/jpeg/meson.build new file mode 100644 index 00000000..8606acc4 --- /dev/null +++ b/src/android/jpeg/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: CC0-1.0 + +android_hal_sources += files([ + 'exif.cpp', + 'post_processor_jpeg.cpp']) + +platform = get_option('android_platform') +if platform == 'generic' + android_hal_sources += files(['encoder_libjpeg.cpp', + 'generic_post_processor_jpeg.cpp', + 'thumbnailer.cpp']) +elif platform == 'cros' + android_hal_sources += files(['cros_post_processor_jpeg.cpp', + 'encoder_jea.cpp']) + android_deps += [dependency('libcros_camera')] +endif diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index d72ebc3c..7ceba60e 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -9,10 +9,10 @@ #include +#include "../android_framebuffer.h" #include "../camera_device.h" #include "../camera_metadata.h" #include "../camera_request.h" -#include "encoder_libjpeg.h" #include "exif.h" #include @@ -44,60 +44,11 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg, streamSize_ = outCfg.size; - thumbnailer_.configure(inCfg.size, inCfg.pixelFormat); - - encoder_ = std::make_unique(); + SetEncoder(); return encoder_->configure(inCfg); } -void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, - const Size &targetSize, - unsigned int quality, - std::vector *thumbnail) -{ - /* Stores the raw scaled-down thumbnail bytes. */ - std::vector rawThumbnail; - - thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail); - - StreamConfiguration thCfg; - thCfg.size = targetSize; - thCfg.pixelFormat = thumbnailer_.pixelFormat(); - int ret = thumbnailEncoder_.configure(thCfg); - - if (!rawThumbnail.empty() && !ret) { - /* - * \todo Avoid value-initialization of all elements of the - * vector. - */ - thumbnail->resize(rawThumbnail.size()); - - /* - * Split planes manually as the encoder expects a vector of - * planes. - * - * \todo Pass a vector of planes directly to - * Thumbnailer::createThumbnailer above and remove the manual - * planes split from here. - */ - std::vector> thumbnailPlanes; - const PixelFormatInfo &formatNV12 = PixelFormatInfo::info(formats::NV12); - size_t yPlaneSize = formatNV12.planeSize(targetSize, 0); - size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1); - thumbnailPlanes.push_back({ rawThumbnail.data(), yPlaneSize }); - thumbnailPlanes.push_back({ rawThumbnail.data() + yPlaneSize, uvPlaneSize }); - - int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes, - *thumbnail, {}, quality); - thumbnail->resize(jpeg_size); - - LOG(JPEG, Debug) - << "Thumbnail compress returned " - << jpeg_size << " bytes"; - } -} - void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) { ASSERT(encoder_); @@ -164,8 +115,8 @@ void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu if (thumbnailSize != Size(0, 0)) { std::vector thumbnail; - generateThumbnail(source, thumbnailSize, quality, &thumbnail); - if (!thumbnail.empty()) + ret = encoder_->generateThumbnail(source, thumbnailSize, quality, &thumbnail); + if (ret > 0 && !thumbnail.empty()) exif.setThumbnail(thumbnail, Exif::Compression::JPEG); } @@ -194,8 +145,7 @@ void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu const uint8_t quality = ret ? *entry.data.u8 : 95; resultMetadata->addEntry(ANDROID_JPEG_QUALITY, quality); - int jpeg_size = encoder_->encode(source, destination->plane(0), - exif.data(), quality); + int jpeg_size = encoder_->encode(streamBuffer, exif.data(), quality); if (jpeg_size < 0) { LOG(JPEG, Error) << "Failed to encode stream image"; processComplete.emit(streamBuffer, PostProcessor::Status::Error); diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index 98309b01..a09f8798 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -8,11 +8,11 @@ #pragma once #include "../post_processor.h" -#include "encoder_libjpeg.h" -#include "thumbnailer.h" #include +#include "encoder.h" + class CameraDevice; class PostProcessorJpeg : public PostProcessor @@ -25,14 +25,9 @@ public: void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override; private: - void generateThumbnail(const libcamera::FrameBuffer &source, - const libcamera::Size &targetSize, - unsigned int quality, - std::vector *thumbnail); + void SetEncoder(); CameraDevice *const cameraDevice_; std::unique_ptr encoder_; libcamera::Size streamSize_; - EncoderLibJpeg thumbnailEncoder_; - Thumbnailer thumbnailer_; }; diff --git a/src/android/meson.build b/src/android/meson.build index 75b4bf20..026b8b3c 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -38,6 +38,7 @@ endif android_deps += [libyuv_dep] android_hal_sources = files([ + 'android_framebuffer.cpp', 'camera3_hal.cpp', 'camera_capabilities.cpp', 'camera_device.cpp', @@ -47,10 +48,6 @@ android_hal_sources = files([ 'camera_ops.cpp', 'camera_request.cpp', 'camera_stream.cpp', - 'jpeg/encoder_libjpeg.cpp', - 'jpeg/exif.cpp', - 'jpeg/post_processor_jpeg.cpp', - 'jpeg/thumbnailer.cpp', 'yuv/post_processor_yuv.cpp' ]) @@ -58,6 +55,7 @@ android_cpp_args = [] subdir('cros') subdir('mm') +subdir('jpeg') android_camera_metadata_sources = files([ 'metadata/camera_metadata.c', diff --git a/src/android/mm/cros_frame_buffer_allocator.cpp b/src/android/mm/cros_frame_buffer_allocator.cpp index 52e8c180..163c5d75 100644 --- a/src/android/mm/cros_frame_buffer_allocator.cpp +++ b/src/android/mm/cros_frame_buffer_allocator.cpp @@ -14,6 +14,7 @@ #include "libcamera/internal/framebuffer.h" +#include "../android_framebuffer.h" #include "../camera_device.h" #include "../frame_buffer_allocator.h" #include "cros-camera/camera_buffer_manager.h" @@ -47,11 +48,11 @@ public: { } - std::unique_ptr + std::unique_ptr allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage); }; -std::unique_ptr +std::unique_ptr PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage) @@ -80,9 +81,11 @@ PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat, plane.length = cros::CameraBufferManager::GetPlaneSize(handle, i); } - return std::make_unique( - std::make_unique(std::move(scopedHandle)), - planes); + auto fb = std::make_unique(handle, + std::make_unique(std::move(scopedHandle)), + planes); + + return fb; } PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION diff --git a/src/android/mm/generic_frame_buffer_allocator.cpp b/src/android/mm/generic_frame_buffer_allocator.cpp index acb2fa2b..c79b7b10 100644 --- a/src/android/mm/generic_frame_buffer_allocator.cpp +++ b/src/android/mm/generic_frame_buffer_allocator.cpp @@ -18,6 +18,7 @@ #include #include +#include "../android_framebuffer.h" #include "../camera_device.h" #include "../frame_buffer_allocator.h" @@ -77,7 +78,7 @@ public: ~Private() override; - std::unique_ptr + std::unique_ptr allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage); private: @@ -92,7 +93,7 @@ PlatformFrameBufferAllocator::Private::~Private() gralloc_close(allocDevice_); } -std::unique_ptr +std::unique_ptr PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage) @@ -135,9 +136,9 @@ PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat, offset += planeSize; } - return std::make_unique( - std::make_unique(allocDevice_, handle), - planes); + return std::make_unique(handle, + std::make_unique(allocDevice_, handle), + planes); } PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION