From patchwork Fri Oct 2 10:34:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 9905 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 013BEC3B5C for ; Fri, 2 Oct 2020 10:34:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3C026035E; Fri, 2 Oct 2020 12:34:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=uajain.com header.i=@uajain.com header.b="UW2P319h"; dkim-atps=neutral Received: from mail.uajain.com (static.126.159.217.95.clients.your-server.de [95.217.159.126]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8DBB06035D for ; Fri, 2 Oct 2020 12:34:24 +0200 (CEST) From: Umang Jain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=uajain.com; s=mail; t=1601634864; bh=zCGdyqe3kkvlrz9FbC68Dra0+l+7Uhw+EvRW0SbDDA0=; h=From:To:Cc:Subject:In-Reply-To:References; b=UW2P319hahjwtn6Ox5RzI1OgEpAxSB1pLpXYV7ZwWXTRjJJUoukaZaKpimf1SEMHT y8ufOraDKnH/5gi6r9kS/Xa7WwTStG2qtLnD474xLl5f/Q/T3XCR8lSpGjMOHt0tlD wkF+YBxNs3mvu/ko227AfdgH6In+v1H/8vSCjG/qiCo0dmlFm4BSgWd5AiO3ISVDcs teFj5g3xB7pRvEX0CiOLo7BwnhcN+jkbVyLVCBAKVdlPL/tWFVLlG+1zqe0l8uadEo pWMEcjbd6XV6TiIA0d8W+CtbIazRVDFbwkX7WMdjf+UdcDEgz2n6vVZqosWQicPFnr dBrPmh2a0OEQQ== To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Oct 2020 16:04:14 +0530 Message-Id: <20201002103416.53623-2-email@uajain.com> In-Reply-To: <20201002103416.53623-1-email@uajain.com> References: <20201002103416.53623-1-email@uajain.com> Mime-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/3] android: jpeg: Add a basic NV12 image thumbnailer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a basic image thumbnailer for NV12 frames being captured. It shall generate a thumbnail image to be embedded as a part of EXIF metadata of the frame. The output of the thumbnail will still be NV12. Signed-off-by: Umang Jain --- src/android/jpeg/encoder_libjpeg.cpp | 12 +++ src/android/jpeg/thumbnailer.cpp | 106 +++++++++++++++++++++++++++ src/android/jpeg/thumbnailer.h | 34 +++++++++ src/android/meson.build | 1 + 4 files changed, 153 insertions(+) create mode 100644 src/android/jpeg/thumbnailer.cpp create mode 100644 src/android/jpeg/thumbnailer.h diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp index 510613c..7d097c6 100644 --- a/src/android/jpeg/encoder_libjpeg.cpp +++ b/src/android/jpeg/encoder_libjpeg.cpp @@ -6,6 +6,7 @@ */ #include "encoder_libjpeg.h" +#include "thumbnailer.h" #include #include @@ -214,6 +215,17 @@ int EncoderLibJpeg::encode(const FrameBuffer *source, LOG(JPEG, Debug) << "JPEG Encode Starting:" << compress_.image_width << "x" << compress_.image_height; + Thumbnailer thScaler; + libcamera::Span thumbnail; + thScaler.configure(Size (compress_.image_width, compress_.image_height), + pixelFormatInfo_->format); + thScaler.scaleBuffer(source, thumbnail); + /* + * \todo: Discuss if we require scaling here or one level above where + * exif data is set (setThumbnail()?). + * Setting thumbnailed scaled data seems a bit convulated initially. + */ + if (nv_) compressNV(&frame); else diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp new file mode 100644 index 0000000..d706ea6 --- /dev/null +++ b/src/android/jpeg/thumbnailer.cpp @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * thumbnailer.cpp - Basic image thumbnailer from NV12 + */ + +#include "thumbnailer.h" +#include "system/graphics.h" + +#include "libcamera/internal/file.h" +#include "libcamera/internal/log.h" + +using namespace libcamera; + +LOG_DEFINE_CATEGORY(Thumbnailer) + +Thumbnailer::Thumbnailer() + : validConfiguration_(false) +{ +} + +void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat) +{ + sourceSize_ = sourceSize; + pixelFormat_ = pixelFormat; + + if (pixelFormat_ .fourcc() == HAL_PIXEL_FORMAT_YV12) { + LOG (Thumbnailer, Error) << "Failed to configure: Pixel Format " + << pixelFormat_.toString() << " unsupported."; + return; + } + + validConfiguration_ = true; +} + +/* Compute a thumbnail size with same aspect ratio as source. */ +void Thumbnailer::computeThumbnailSize() +{ + unsigned int baseWidth = 160; + + unsigned int width = baseWidth * sourceSize_.width / sourceSize_.height; + targetSize_.width = width - (width % 16); + targetSize_.height = targetSize_.width * sourceSize_.height + / sourceSize_.width; + if (targetSize_.height & 1) + targetSize_.height++; +} + +int Thumbnailer::scaleBuffer(const FrameBuffer *source, Span &dest) +{ + MappedFrameBuffer frame(source, PROT_READ); + if (!frame.isValid()) { + LOG(Thumbnailer, Error) << "Failed to map FrameBuffer : " + << strerror(frame.error()); + return frame.error(); + } + + if (!validConfiguration_) { + LOG(Thumbnailer, Error) << "config is unconfigured or invalid."; + return -1; + } + + computeThumbnailSize(); + + const unsigned int sw = sourceSize_.width; + const unsigned int sh = sourceSize_.height; + const unsigned int tw = targetSize_.width; + const unsigned int th = targetSize_.height; + + /* Image scaling block implementing nearest-neighbour algorithm. */ + unsigned char *src = static_cast(frame.maps()[0].data()); + unsigned char *src_c = src + sh * sw; + unsigned int cb_pos = 0; + unsigned int cr_pos = 1; + unsigned char *src_cb, *src_cr; + + size_t dstSize = (th * tw) + ((th/2) * tw); + unsigned char *destination = static_cast(malloc(dstSize)); + unsigned char *dst = destination; + unsigned char *dst_c = destination + th * tw; + + for (unsigned int y = 0; y < th; y+=2) { + unsigned int sourceY = (sh*y + th/2) / th; + + src_cb = src_c + (sourceY/2) * sw + cb_pos; + src_cr = src_c + (sourceY/2) * sw + cr_pos; + + for (unsigned int x = 0; x < tw; x+=2) { + unsigned int sourceX = (sw*x + tw/2) / tw; + + dst[y * tw + x] = src[sw * sourceY + sourceX]; + dst[(y+1) * tw + x] = src[sw * (sourceY+1) + sourceX]; + dst[y * tw + (x+1)] = src[sw * sourceY + (sourceX+1)]; + dst[(y+1) * tw + (x+1)] = src[sw * (sourceY+1) + (sourceX+1)]; + + dst_c[(y/2) * tw + x + cb_pos] = src_cb[(sourceX/2) * 2]; + dst_c[(y/2) * tw + x + cr_pos] = src_cr[(sourceX/2) * 2]; + } + } + + /* Write scaled pixels to dest */ + dest = { destination, dstSize }; + + return 0; +} diff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h new file mode 100644 index 0000000..8ba9816 --- /dev/null +++ b/src/android/jpeg/thumbnailer.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * thumbnailer.h - Basic image thumbnailer from NV12 + */ +#ifndef __ANDROID_JPEG_THUMBNAILER_H__ +#define __ANDROID_JPEG_THUMBNAILER_H__ + +#include + +#include "libcamera/internal/buffer.h" +#include "libcamera/internal/formats.h" + +class Thumbnailer +{ +public: + Thumbnailer(); + + void configure(const libcamera::Size &sourceSize, + libcamera::PixelFormat pixelFormat); + int scaleBuffer(const libcamera::FrameBuffer *source, libcamera::Span &dest); + +private: + void computeThumbnailSize(); + + libcamera::PixelFormat pixelFormat_; + libcamera::Size sourceSize_; + libcamera::Size targetSize_; + + bool validConfiguration_; +}; + +#endif /* __ANDROID_JPEG_THUMBNAILER_H__ */ diff --git a/src/android/meson.build b/src/android/meson.build index 0293c20..f35ba04 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -22,6 +22,7 @@ android_hal_sources = files([ 'camera_ops.cpp', 'jpeg/encoder_libjpeg.cpp', 'jpeg/exif.cpp', + 'jpeg/thumbnailer.cpp', ]) android_camera_metadata_sources = files([ From patchwork Fri Oct 2 10:34:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 9906 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 544ABC3B5C for ; Fri, 2 Oct 2020 10:34:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 21A73622B5; Fri, 2 Oct 2020 12:34:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=uajain.com header.i=@uajain.com header.b="Wr8z4r5A"; dkim-atps=neutral Received: from mail.uajain.com (static.126.159.217.95.clients.your-server.de [95.217.159.126]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE20E6035D for ; Fri, 2 Oct 2020 12:34:26 +0200 (CEST) From: Umang Jain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=uajain.com; s=mail; t=1601634866; bh=QiG603B0Vq6LSYuygYIgakiBlTEi43DvdgI2kP1ff0w=; h=From:To:Cc:Subject:In-Reply-To:References; b=Wr8z4r5AxB15jle2CiZLeYPGJXxl+BJyFUXOmM42SfZR7zcEOPlrmK3+kyaYtGTL2 EahrrZw0cKUpa3aqIxfCY/9XB//vlm4L22u3z5TImKWehnv2wKTQUWGTCzCD1giICD IXumbJdFtHD8Zxr4ib1WDvjE32NaThDMKfkd9p2etTCRwiLFl82TesqOPiecUSpvsF nD/+yUqHHsd9V1CjaQC7wdd3q4d7z6fCiUGVyOCdq4jINlJNnlojjPpmiqxbYcuztF NuVUK7qUXJLDdUo3WH0pCopMS4vHlz8eTAHA27GoohCBidsWr08ssX2fMnjcFz5OJU aS+nEVudg/t3w== To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Oct 2020 16:04:15 +0530 Message-Id: <20201002103416.53623-3-email@uajain.com> In-Reply-To: <20201002103416.53623-1-email@uajain.com> References: <20201002103416.53623-1-email@uajain.com> Mime-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/3] [DNI] jpeg/thumbnailer: write thumbnail to file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Umang Jain --- src/android/jpeg/thumbnailer.cpp | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp index d706ea6..d1dbdaf 100644 --- a/src/android/jpeg/thumbnailer.cpp +++ b/src/android/jpeg/thumbnailer.cpp @@ -34,6 +34,22 @@ void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat) validConfiguration_ = true; } +static std::string datetime() +{ + time_t rawtime; + struct tm *timeinfo; + char buffer[80]; + static unsigned int milliseconds = 0; + + time(&rawtime); + timeinfo = localtime(&rawtime); + + strftime(buffer, 80, "%d-%m-%Y.%H-%M-%S.", timeinfo); + + /* milliseconds is just a fast hack to ensure unique filenames */ + return std::string(buffer) + std::to_string(milliseconds++); +} + /* Compute a thumbnail size with same aspect ratio as source. */ void Thumbnailer::computeThumbnailSize() { @@ -99,6 +115,12 @@ int Thumbnailer::scaleBuffer(const FrameBuffer *source, Span &dest) } } + /* Helper code: Write the output pixels to a file so we can inspect */ + File file("/tmp/" + datetime() + ".raw"); + int32_t ret = file.open(File::WriteOnly); + ret = file.write({ destination, dstSize }); + LOG(Thumbnailer, Info) << "Wrote " << ret << " bytes: " << targetSize_.width << "x" << targetSize_.height; + /* Write scaled pixels to dest */ dest = { destination, dstSize }; From patchwork Fri Oct 2 10:34:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 9907 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5380C3B5C for ; Fri, 2 Oct 2020 10:34:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7479763B2A; Fri, 2 Oct 2020 12:34:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=uajain.com header.i=@uajain.com header.b="jhFwNoW/"; dkim-atps=neutral Received: from mail.uajain.com (static.126.159.217.95.clients.your-server.de [95.217.159.126]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 949276035D for ; Fri, 2 Oct 2020 12:34:29 +0200 (CEST) From: Umang Jain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=uajain.com; s=mail; t=1601634869; bh=FEtRYcTQyqtjJVRf8peI8iYjMXb3nWgf8n3fk4TttBI=; h=From:To:Cc:Subject:In-Reply-To:References; b=jhFwNoW/bszkMnjcn6Q8MdabnUj2gvifpptDGgkABmQ8/KWwhDV+/8L826Zovas3f rYEx906KwVG0+qXp3avo7be3wEBB6xDYgJ0paVLMldV1p4jhvM9fBY75ZE7hyzs5WE +3b6t062rOdpEM6tr427wgxVyWtKLkexTgI34el962dz4te/+Yd+zJt3zq4OLfOtwE IwLBJrZXv5ZnPhuHrnmMa3nRZMrmsmuJer0JEepOUyzjlsFqY5K8KzqZzn+Cto6A0T mR31/T8iXhOqAmYqvfmySlK6nYTxoA13eUdwJlqBoDeBlHbssqigiz1zApd3lfVrp/ U0IGueiheAaLA== To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Oct 2020 16:04:16 +0530 Message-Id: <20201002103416.53623-4-email@uajain.com> In-Reply-To: <20201002103416.53623-1-email@uajain.com> References: <20201002103416.53623-1-email@uajain.com> Mime-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 3/3] [DNI] Add HOWTO scaler instructions for future reference X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" --- scaler-instructions.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 scaler-instructions.txt diff --git a/scaler-instructions.txt b/scaler-instructions.txt new file mode 100644 index 0000000..3151db3 --- /dev/null +++ b/scaler-instructions.txt @@ -0,0 +1,18 @@ +# Build uajain/exif-scaler-NV12 tester with -Dandroid=enabled +# Load the vivid module if not done already + +# Capture few frames in NV12 from cam +./build/src/cam/cam -c vivid -C2 -F -s pixelformat=NV12 + +# Note down scaled down image's size from the above capture's o/p $size +# scaled down raw output is written to /tmp + +cd /tmp +~/src/raw2rgbpnm/raw2rgbpnm -s $size -f RGB24 30-09-2020.18-47-29.0.raw thumbnail.ppm + +# View new.ppm +eog thumbnail.ppm + +--- + +raw2rgbpnm git : git://git.retiisi.eu/~sailus/raw2rgbpnm.git