From patchwork Thu Jan 28 22:42:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11046 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6605DBD808 for ; Thu, 28 Jan 2021 22:42:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34B4768397; Thu, 28 Jan 2021 23:42:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="aDhr12tb"; dkim-atps=neutral Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2207F68392 for ; Thu, 28 Jan 2021 23:42:26 +0100 (CET) Received: by mail-pg1-x529.google.com with SMTP id z21so5289317pgj.4 for ; Thu, 28 Jan 2021 14:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=870VFbxCNzznAQu57QWTH6FxCHRak+Ro7c8lj5zTWrU=; b=aDhr12tbhGSPaknqAX0tv9Pkgbz67I9n6OUYjleW7NCC5ZM0KVXXs5flFWqPL3NprJ xCi/veDrWlHg+bzUzp0BkNgaLHYuZ+4/iyouLAkO4y9Ne85vYKsgAXFgM6KvNk4MNBuT rHdOi6V56yP9uKRDTNjAhaZSnJ6PNSgaVn97w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=870VFbxCNzznAQu57QWTH6FxCHRak+Ro7c8lj5zTWrU=; b=B6EezuAhZmx8FVZzXpnLaPJJyoJJgFKMyDw4pAIdX1GIW+9ep3G6udoPph90I7FWDI iT+RVK/gthv7fOKh331OUDCxUIEwf5G1zuuEDg3+zJi7WL+ZTIRi5BbL2A9WDyX4M8ur Y6WVrSxm1IWt7Icz3T+18Bp9qtKdPbL72PIs+Z3WsjVun2CkIXWizvrVrJI3vXpIt1nE lonVEVH0kXDW0ZykDXnL1TSmQ5Ij8RWCO0d32QhdAWDppi9ltz5dHMBM3+TATHnLRJ4j SaFR4TYDxzplRHwK5v6hu1HnpGHrUrYeSPA0M+3Ouj5W+Uy+rEjbrwcHJQWgfKZ3TIJd Nx8A== X-Gm-Message-State: AOAM531O39Bzl+4jkWG1mgHrNwvpCxcIAeAXqTzHIxyXn7o8l/Qm95+j t0Fed+jph6RYn3LF7yqRtsyCkML7LIwr/0My X-Google-Smtp-Source: ABdhPJz9l9VlqCwjAR2HZENeAtS4gWXtEdDU+UCxFuSx85a6c0TVni7zOvgbMnQOHZn+SQoof/Ez1A== X-Received: by 2002:a63:3303:: with SMTP id z3mr1586846pgz.302.1611873744541; Thu, 28 Jan 2021 14:42:24 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id md7sm5866288pjb.52.2021.01.28.14.42.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Jan 2021 14:42:23 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 28 Jan 2021 22:42:15 +0000 Message-Id: <20210128224217.2919790-2-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210128224217.2919790-1-hiroh@chromium.org> References: <20210128224217.2919790-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/2] android: post_processor: Change the type destination in process() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The type of the destination buffer in PostProcessor::process() is libcamera::Span. libcamera::Span is used for one dimension buffer (e.g. blob buffer). The destination can be multiple dimensions buffer (e.g. yuv frame). Therefore, this changes the type of the destination buffer to MappedFrameBuffer. Signed-off-by: Hirokazu Honda Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- src/android/camera_stream.cpp | 4 ++-- src/android/camera_stream.h | 6 ++++-- src/android/jpeg/post_processor_jpeg.cpp | 6 +++--- src/android/jpeg/post_processor_jpeg.h | 2 +- src/android/post_processor.h | 7 ++++--- 5 files changed, 14 insertions(+), 11 deletions(-) -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 4c8020e5..611ec0d1 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -96,14 +96,14 @@ int CameraStream::configure() } int CameraStream::process(const libcamera::FrameBuffer &source, - MappedCamera3Buffer *dest, + libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) { if (!postProcessor_) return 0; - return postProcessor_->process(source, dest->maps()[0], + return postProcessor_->process(source, destination, requestMetadata, resultMetadata); } diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 298ffbf6..73bac0ba 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -19,9 +19,10 @@ #include #include +#include + class CameraDevice; class CameraMetadata; -class MappedCamera3Buffer; class PostProcessor; class CameraStream @@ -119,9 +120,10 @@ public: int configure(); int process(const libcamera::FrameBuffer &source, - MappedCamera3Buffer *dest, + libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata); + libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index cac0087b..74e81c6f 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -83,7 +83,7 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, } int PostProcessorJpeg::process(const FrameBuffer &source, - Span destination, + libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) { @@ -171,8 +171,8 @@ int PostProcessorJpeg::process(const FrameBuffer &source, ret = requestMetadata.getEntry(ANDROID_JPEG_QUALITY, &entry); const uint8_t quality = ret ? *entry.data.u8 : 95; resultMetadata->addEntry(ANDROID_JPEG_QUALITY, &quality, 1); + int jpeg_size = encoder_->encode(source, destination->maps()[0], exif.data(), quality); - int jpeg_size = encoder_->encode(source, destination, exif.data(), quality); if (jpeg_size < 0) { LOG(JPEG, Error) << "Failed to encode stream image"; return jpeg_size; @@ -190,7 +190,7 @@ int PostProcessorJpeg::process(const FrameBuffer &source, * \todo Investigate if the buffer size mismatch is an issue or * expected behaviour. */ - uint8_t *resultPtr = destination.data() + + uint8_t *resultPtr = destination->maps()[0].data() + cameraDevice_->maxJpegBufferSize() - sizeof(struct camera3_jpeg_blob); auto *blob = reinterpret_cast(resultPtr); diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index d2dfa450..7689de73 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -25,7 +25,7 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; int process(const libcamera::FrameBuffer &source, - libcamera::Span destination, + libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) override; diff --git a/src/android/post_processor.h b/src/android/post_processor.h index bda93bb4..f5c99f03 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -8,9 +8,10 @@ #define __ANDROID_POST_PROCESSOR_H__ #include -#include #include +#include + class CameraMetadata; class PostProcessor @@ -21,9 +22,9 @@ public: virtual int configure(const libcamera::StreamConfiguration &inCfg, const libcamera::StreamConfiguration &outCfg) = 0; virtual int process(const libcamera::FrameBuffer &source, - libcamera::Span destination, + libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) = 0; + CameraMetadata *metadata) = 0; }; #endif /* __ANDROID_POST_PROCESSOR_H__ */ From patchwork Thu Jan 28 22:42:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11047 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D5E5BD808 for ; Thu, 28 Jan 2021 22:42:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CFFA968396; Thu, 28 Jan 2021 23:42:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VYv5JfnV"; dkim-atps=neutral Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B47856838E for ; Thu, 28 Jan 2021 23:42:27 +0100 (CET) Received: by mail-pj1-x1034.google.com with SMTP id l18so5250978pji.3 for ; Thu, 28 Jan 2021 14:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Sb/64puLYPXfOeUQQBdbMmKpH6LeVFA1Wk43ULvbUFg=; b=VYv5JfnVKEBJdW3At7mTNS8N4QszW2VvwqTvWHr7w71gdUdNHBVVpsrEuFyG18DMTs WGfGdAyR4u3RSsgnXLwm3b/eO/mqGSMwg1N0Yse+jhNh4G0VrbyA2nUfvud11BZpDt/H wPSLlUz72WMxQZERv77HU5hFBhG4gTWaEkxLA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Sb/64puLYPXfOeUQQBdbMmKpH6LeVFA1Wk43ULvbUFg=; b=ulSd4yO3O9IrxBRdfPpS/aF1g25u30lX8ayayn6HNZE9Urp7K6HeZ+qFqLHed2Is0b F5kSPNsb2PjDa7J7lIKdnHLXeb7M6jSDoyXIW8DvqVVAdDJ4YzbIyOCClcPNTE3Rgyz2 JDtfY3nH5SqZzXtV17ipY61VPIbdesIs7kfMu0M/iVUhVL9IjlBWbCMAqqqLQrlZevF9 e1uwOftHA29cVaXHErGI3v9XVGGyQ17Pc/KH0Ef8s72yJXC0+2v+Dw6aV39HhkLSsbFj umFek8dguGH2qYIXhQx+DpqWzzl8OKQo5CZbmDcHRXNFJhAZcHnYsarA5e81ip4y5MnY xXFw== X-Gm-Message-State: AOAM5312rlTuIB7/PZhgWDJSIcOk/hcvXNhRaockFCZLNqD32ukUkES8 8ZpHWH2zHXUAMqbcGHB6pgLJGBEC+N+MLoc+ X-Google-Smtp-Source: ABdhPJyaGOKiD0F++j5MGT0kjVZTAsEBMCN4Af8LLe34KwEuWCs9Qwz3m+k7xPP9rhvXpAdRxqNnpg== X-Received: by 2002:a17:902:bb90:b029:df:bdbb:be99 with SMTP id m16-20020a170902bb90b02900dfbdbbbe99mr1358500pls.52.1611873745957; Thu, 28 Jan 2021 14:42:25 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id md7sm5866288pjb.52.2021.01.28.14.42.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Jan 2021 14:42:25 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 28 Jan 2021 22:42:16 +0000 Message-Id: <20210128224217.2919790-3-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210128224217.2919790-1-hiroh@chromium.org> References: <20210128224217.2919790-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/2] android: libyuv: Introduce PostProcessorLibyuv X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds PostProcessorLibyuv. It supports NV12 buffer scaling. Signed-off-by: Hirokazu Honda --- src/android/libyuv/post_processor_libyuv.cpp | 123 +++++++++++++++++++ src/android/libyuv/post_processor_libyuv.h | 44 +++++++ src/android/meson.build | 1 + 3 files changed, 168 insertions(+) create mode 100644 src/android/libyuv/post_processor_libyuv.cpp create mode 100644 src/android/libyuv/post_processor_libyuv.h -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/libyuv/post_processor_libyuv.cpp b/src/android/libyuv/post_processor_libyuv.cpp new file mode 100644 index 00000000..81f237e0 --- /dev/null +++ b/src/android/libyuv/post_processor_libyuv.cpp @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * post_processor_libyuv.cpp - Post Processor using libyuv + */ + +#include "post_processor_libyuv.h" + +#include + +#include +#include +#include +#include +#include + +using namespace libcamera; + +LOG_DEFINE_CATEGORY(LIBYUV) + +int PostProcessorLibyuv::configure(const StreamConfiguration &inCfg, + const StreamConfiguration &outCfg) +{ + if (inCfg.pixelFormat != outCfg.pixelFormat) { + LOG(LIBYUV, Error) + << "Pixel format conversion is not supported" + << " (from " << inCfg.toString() + << " to " << outCfg.toString() << ")"; + return -EINVAL; + } + + if (inCfg.size < outCfg.size) { + LOG(LIBYUV, Error) << "Up-scaling is not supported" + << " (from " << inCfg.toString() + << " to " << outCfg.toString() << ")"; + return -EINVAL; + } + + if (inCfg.pixelFormat == formats::NV12) { + LOG(LIBYUV, Error) << "Unsupported format " << inCfg.pixelFormat + << " (only NV12 is supported)"; + return -EINVAL; + } + + getNV12LengthAndStride(inCfg.size, sourceLength_, sourceStride_); + getNV12LengthAndStride(outCfg.size, destinationLength_, + destinationStride_); + return 0; +} + +int PostProcessorLibyuv::process(const FrameBuffer &source, + libcamera::MappedBuffer *destination, + [[maybe_unused]] const CameraMetadata &requestMetadata, + [[maybe_unused]] CameraMetadata *metadata) +{ + if (!isValidNV12Buffers(source, *destination)) + return -EINVAL; + + const MappedFrameBuffer sourceMapped(&source, PROT_READ); + if (!sourceMapped.isValid()) { + LOG(LIBYUV, Error) << "Failed to mmap camera frame buffer"; + return -EINVAL; + } + + return 0 == libyuv::NV12Scale(sourceMapped.maps()[0].data(), + sourceStride_[0], + sourceMapped.maps()[1].data(), + sourceStride_[1], + sourceSize_.width, sourceSize_.height, + destination->maps()[0].data(), + destinationStride_[0], + destination->maps()[1].data(), + destinationStride_[1], + destinationSize_.width, + destinationSize_.height, + libyuv::FilterMode::kFilterBilinear); +} + +bool PostProcessorLibyuv::isValidNV12Buffers( + const FrameBuffer &source, + const libcamera::MappedBuffer &destination) const +{ + if (source.planes().size() != 2u) { + LOG(LIBYUV, Error) << "The number of source planes is not 2"; + return false; + } + if (destination.maps().size() != 2u) { + LOG(LIBYUV, Error) + << "The number of destination planes is not 2"; + return false; + } + + if (source.planes()[0].length < sourceLength_[0] || + source.planes()[1].length < sourceLength_[1]) { + LOG(LIBYUV, Error) + << "The source planes lengths are too small"; + return false; + } + if (destination.maps()[0].size() < destinationLength_[0] || + destination.maps()[1].size() < destinationLength_[1]) { + LOG(LIBYUV, Error) + << "The destination planes lengths are too small"; + return false; + } + + return true; +} + +// static +void PostProcessorLibyuv::getNV12LengthAndStride(Size size, + unsigned int length[2], + unsigned int stride[2]) +{ + const PixelFormatInfo &nv12Info = PixelFormatInfo::info(formats::NV12); + for (unsigned int i = 0; i < 2; i++) { + const unsigned int vertSubSample = + nv12Info.planes[i].verticalSubSampling; + length[i] = nv12Info.stride(size.width, i, /*align=*/1) * + ((size.height + vertSubSample - 1) / vertSubSample); + stride[i] = nv12Info.stride(size.width, i, 1); + } +} diff --git a/src/android/libyuv/post_processor_libyuv.h b/src/android/libyuv/post_processor_libyuv.h new file mode 100644 index 00000000..a65a16b3 --- /dev/null +++ b/src/android/libyuv/post_processor_libyuv.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * post_processor_libyuv.h - Post Processor using libyuv + */ +#ifndef __ANDROID_POST_PROCESSOR_LIBYUV_H__ +#define __ANDROID_POST_PROCESSOR_LIBYUV_H__ + +#include + +#include "../post_processor.h" + +class CameraDevice; + +class PostProcessorLibyuv : public PostProcessor +{ +public: + PostProcessorLibyuv() = default; + + int configure(const libcamera::StreamConfiguration &incfg, + const libcamera::StreamConfiguration &outcfg) override; + int process(const libcamera::FrameBuffer &source, + libcamera::MappedBuffer *destination, + const CameraMetadata & /*requestMetadata*/, + CameraMetadata *metadata) override; + +private: + bool isValidNV12Buffers(const libcamera::FrameBuffer &source, + const libcamera::MappedBuffer &destination) const; + + static void getNV12LengthAndStride(libcamera::Size size, + unsigned int length[2], + unsigned int stride[2]); + + libcamera::Size sourceSize_; + libcamera::Size destinationSize_; + unsigned int sourceLength_[2] = {}; + unsigned int destinationLength_[2] = {}; + unsigned int sourceStride_[2] = {}; + unsigned int destinationStride_[2] = {}; +}; + +#endif /* __ANDROID_POST_PROCESSOR_LIBYUV_H__ */ diff --git a/src/android/meson.build b/src/android/meson.build index e1533d7c..cbe09200 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -44,6 +44,7 @@ android_hal_sources = files([ 'jpeg/exif.cpp', 'jpeg/post_processor_jpeg.cpp', 'jpeg/thumbnailer.cpp', + 'libyuv/post_processor_libyuv.cpp' ]) android_camera_metadata_sources = files([