From patchwork Tue Jul 21 22:01:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 8909 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 543A3C2E68 for ; Tue, 21 Jul 2020 22:01:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4788F60923; Wed, 22 Jul 2020 00:01:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t8AoyMpO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4978060491 for ; Wed, 22 Jul 2020 00:01:31 +0200 (CEST) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A42BD51A; Wed, 22 Jul 2020 00:01:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1595368890; bh=fr753PagWGrk1/ofTGu8Yrj5ZTpzWxpVg1F03B3kzcE=; h=From:To:Cc:Subject:Date:From; b=t8AoyMpO6oxShhAJGc6dp6hnz02J+i2eKxz+ccrh6WGspMl4myfb1T83ypqBfLWQp X9wdHtPTk5K7CHGtvh4VkfINHMPjiLTWGZsrP7pKPBZ91+JmBrcbCDFtDtorwJ4Alq M4/3X4aEBIUSi0XjkM0WH7aDWzDAqmMRqbsX0aOc= From: Kieran Bingham To: libcamera devel Date: Tue, 21 Jul 2020 23:01:20 +0100 Message-Id: <20200721220126.202065-1-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 0/6] android: jpeg / software streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These patches are posted just for discussion points, they are most certainly unfinished, and in a state of flux, but plumbing multiple streams or software streams is becoming a blocker to others, and so to is the lack of being able to run the Camera application. With these patches, at least the camera application can run, albeit not successfully write out a captured image yet. The compression of the image is successfull, and has been tested by directly writing the compressed data to temporary files. At least the following is known to still warrant more work: - JPEG compression should use the raw interfaces to reduce conversions I hope to work on this next, but I have been keen to get the plumbing working, so performance improvements can happen after. - Error handling from libjpeg needs improving, and a custom error handler implementing. - I was going to strip out the non NV12 conversion functions, for rfc, but they're not actually that big. But they should be replaced by the raw interface where possible. - Added software streams likely need a validation cycle to determine what can be achieved by the libcamera streams. - Ideally the JPEG_MAX_SIZE should come from libjpeg, but that's a bit chicken and egg, as the JPEG_MAX_SIZE is a static property before we have knowledge of what stream size we will encode. - There's debug prints everywhere ;-) - It would be nice to not have to duplicate the PixelFormat information tables. - The JPEG compressor isn't actaully destructed yet, and I think there will be benefit to more refactoring of the CameraStream class and perhaps moving code to the CameraStream to handle actual frame processing. - The JPEG compression should be moved to happen in it's own thread to prevent blocking of any other processing. Kieran Bingham (6): android: Introduce JPEG compression android: camera_device: Report configuration changes from validate() android: camera_device: Add ANDROID_JPEG_MAX_SIZE android: camera_device: Support MJPEG stream construction [HACK]: android: camera_device: Define JPEG size and quality android: camera_device: Provide a MappedCamera3Buffer src/android/camera_device.cpp | 169 +++++++++++++++- src/android/camera_device.h | 8 + src/android/jpeg/compressor.h | 28 +++ src/android/jpeg/compressor_jpeg.cpp | 279 +++++++++++++++++++++++++++ src/android/jpeg/compressor_jpeg.h | 44 +++++ src/android/meson.build | 1 + src/libcamera/meson.build | 2 + 7 files changed, 522 insertions(+), 9 deletions(-) create mode 100644 src/android/jpeg/compressor.h create mode 100644 src/android/jpeg/compressor_jpeg.cpp create mode 100644 src/android/jpeg/compressor_jpeg.h