From patchwork Wed Oct 20 10:42:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14188 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6568BBF415 for ; Wed, 20 Oct 2021 10:42:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D574E68F59; Wed, 20 Oct 2021 12:42:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DqESYLNc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F41D6023A for ; Wed, 20 Oct 2021 12:42:19 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.185]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 68AAB2A5; Wed, 20 Oct 2021 12:42:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634726539; bh=yorJn/CZyAOakNip+3SbrIMBmfQYrjpFTTG4jIQGyGU=; h=From:To:Cc:Subject:Date:From; b=DqESYLNc9OF4E4GuGFa1B3wDfQhIJn2TWmgIsdFr8fYsVR147sAerN5dNsrn/hekV N3+2eBFPTwqQBKhVAZsTRXZdQHdrQNvKWUgK38MSMszHLYVmiTZKm6OMBe8VSyBnH+ 7sDTRbuUDJy8WctUyEoBY5NfvEV+tE9Z3C4Dj1BY= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 16:12:08 +0530 Message-Id: <20211020104212.121743-1-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 0/4] Async Post Processor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is v5 for Async Post Processor based on top of recent HAL refactoring "android: Overhaul request completion handling" This series primarily introduces a Thread worker class along with a queue, in which request for post-process of streams are queued and processed ansychronously. All the context information is carried inside Camera3RequestDescriptor. Patch 1/4 and 2/4 introduces a notification mechanism to know that a post-processing request has been finished. These run synchronously. 3/4 introduces the asynchronous-nity along with post-processor's queue. 4/4 prevents a small window of race while setting the descriptor status_ (since status can be set from 2 threads now, main thread vs post_processor's thread) No CTS regressions seen on nautilus with this series. Some open questions/discussions: - As flush/stop scenarios are kept out of scope of this series, however, I have kept flush() implementation for PostProcessorWorker class in this series. It isn't called but makes the patch complete to me. - streamsProcessMutex_ currently guards only post-processing bits while it comes to descriptor->buffers_. Should we access-control all calls to descriptor->buffers_ with streamProcessMutex_ (I don't think so) but there might be a race I haven't considered. - As seen by 2/4, CameraStream::process returns a value to fail-check. PostProcessor::process() doesn't (rather it notifies via signal mechanism). There might be a case where multiple streams require post-processor, 1 one them gets queued sucessfully, other one fails, returning error value from CameraStream::process(). How would we handle this situation. There is \todo: mark in 3/4 regarding this. Laurent Pinchart (1): android: camera_device: Protect descriptor status_ with lock Umang Jain (3): android: Notify post processing completion via a signal android: post_processor: Drop return value for process() android: post_processor: Make post processing async src/android/camera_device.cpp | 143 +++++++++++++++++------ src/android/camera_device.h | 12 +- src/android/camera_request.h | 8 ++ src/android/camera_stream.cpp | 122 +++++++++++++++++-- src/android/camera_stream.h | 38 +++++- src/android/jpeg/post_processor_jpeg.cpp | 14 +-- src/android/jpeg/post_processor_jpeg.h | 6 +- src/android/post_processor.h | 15 ++- src/android/yuv/post_processor_yuv.cpp | 20 ++-- src/android/yuv/post_processor_yuv.h | 6 +- 10 files changed, 312 insertions(+), 72 deletions(-)