From patchwork Tue Jul 26 18:24:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanlin Chen X-Patchwork-Id: 16801 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B23E2BE173 for ; Tue, 26 Jul 2022 18:25:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3BCD363312; Tue, 26 Jul 2022 20:25:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658859924; bh=MviuWh4zQGjni2hAzoonNBcAB/MdzVJ3JNGunLLQ4hU=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=zuoX0MhWNpH5Z4J6v2XkLL6S2bp1O0qdpUBsEgaCqfnFrstwIvGmZQMGIpZ4+Bsca /TuK9d6X/+RhW11XQWJZH6LDDhvl8DCyJhsrNCCsT2j11OJRba7TB6O5V1dfirWacU mv/VynIvYRASHsC6szG42djAg5lkDAD/XaJ6/0svH14Jn2q3zzOg8yKMG0AV256UHm y/iD0llbxgRcsay03vX1UahT4B/jvJYDsA/Ghnmy0sexWop2UZkgfTcD9MwLat5st8 TWI3gk9oq1jyE8X27FDDFpswMdoXENn0PRm7ZQGi7tFFaLDcPf+MQa/HFaPDiVhnDA O/gvqEwoCmjaQ== Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1089F60487 for ; Tue, 26 Jul 2022 20:25:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="YLQbTZZI"; dkim-atps=neutral Received: by mail-pj1-x102a.google.com with SMTP id q7-20020a17090a7a8700b001f300db8677so1204236pjf.5 for ; Tue, 26 Jul 2022 11:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dWLbX6R/ZbYmUO9MZF8VUxCzZRrNNcr+kgwNLzBO0e4=; b=YLQbTZZIFw3M6hyDFdBS9VWEt7WLJecQwH2AVBGhsapR1oVNVcWzphqU3U7tFmO6DX /Y9kjH003bmO1BG1NqABhaoNI+kQWS65qmEUlEy/lZz/Xoc7HrZxD5zcH5zB0Ml8rJ0j 2ukWYs3xq0s0CUN1ryH+G6VKcU/icKmkqTo5o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dWLbX6R/ZbYmUO9MZF8VUxCzZRrNNcr+kgwNLzBO0e4=; b=2uPF5a5eyMwHwxUmpWT/RIvO1bKAN2uPKv42OdzOcuvVhVJbs5xRwOlEY35y72dG4s iLpBnZgQ6sfXP6No38Y8jJ1kHRFL84NqT8xo0nDpGTg87JktYdxRy/Ip+NCFMaALqswf Y4/wN9EWr/QK4Tqf5Mg33op2U+koVEHl2MxgNFu1dARp3AUPgBWcZKBKMMOYwWBBYoU6 yglcuWZVyaYhM0l+zo7UdXTx7baEKGJpj6ZDdljd6kO6ad/0s4V6/3YB4/d4AOAyDwMG QKiaDZChhl8K+6VHlgycgxH+ANrPNdE24/CHm7EjA7b5DvTfT8RVLnqoCDVfIxs58LnB BSrg== X-Gm-Message-State: AJIora8trdP7o974WODV78AzlpX30fTmzYcgMgtk26vkgGP0xZ+wqkyK GVVj8uTAHGTfqgBuiTexB3r+ek5Rcqmbng== X-Google-Smtp-Source: AGRyM1v3d0b2ykpuZYNvY0kdHR0D6jvbVvRNrg1Cf7GVehKELv25NQWGJw/UucnVjLyojnQ/gvvwsQ== X-Received: by 2002:a17:902:bb8a:b0:16d:190d:981d with SMTP id m10-20020a170902bb8a00b0016d190d981dmr17958556pls.53.1658859920875; Tue, 26 Jul 2022 11:25:20 -0700 (PDT) Received: from localhost ([2401:fa00:1:17:739:6574:4032:5911]) by smtp.gmail.com with UTF8SMTPSA id l3-20020a170903244300b0015eab1b097dsm3063191pls.22.2022.07.26.11.25.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Jul 2022 11:25:20 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 27 Jul 2022 02:24:54 +0800 Message-Id: <20220726182500.425182-1-hanlinchen@chromium.org> X-Mailer: git-send-email 2.37.1.359.gd136c6c3e2-goog MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 0/6] Implement Android Partial Result Featrue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Han-Lin Chen via libcamera-devel From: Hanlin Chen Reply-To: Han-Lin Chen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The series is to implement Android partial result feature, which allows pipeline handler returns early buffers and metadata back to application before request completed. The feature is as a preparation of implementing the following description of Android camera3 spec: * The order of returning metadata and buffers for a single result does not * matter, but buffers for a given stream must be returned in FIFO order. So * the buffer for request 5 for stream A must always be returned before the * buffer for request 6 for stream A. This also applies to the result * metadata; the metadata for request 5 must be returned before the metadata * for request 6. * * However, different streams are independent of each other, so it is * acceptable and expected that the buffer for request 5 for stream A may be * returned after the buffer for request 6 for stream B is. And it is * acceptable that the result metadata for request 6 for stream B is * returned before the buffer for request 5 for stream A is. The spec is critical for ZSL implementation, because the still capture can be return in an order independent to pending preview requests. This will enhance still capture performance by: 1. Use cached raw frame for still capture, skip pending sensor exposure. 2. Skip pending preview requests and return still capture concurently. 2. HAL can pick 3A stable raw frame, so the application doesn't need to wait for the process of stablizing 3A (by AEPrecapture and AFTrigger). Because Android relies on metadata to process buffer properly, bufferCompleted signal is not sufficient to report partial result to framework. The patch series allows returning metadata earlier for pipeline handler and refactor Android adaptor to return partial result as soon as possible. Han-Lin Chen (6): libcamera: Camera: Add RequestCompletionMode to configure the completion order libcamera: Camera: Add signals for completion of metadata and partial result libcamera: ipu3: returning partial results android: Move StreamBuffer out of Camera3RequestDescriptor android: Add JpegExifMetadata to store tags setting into Exif android: Implement partial result feature based on partialResultCompleted signal include/libcamera/camera.h | 10 + include/libcamera/internal/camera.h | 4 + include/libcamera/internal/pipeline_handler.h | 3 + include/libcamera/request.h | 40 + src/android/camera_capabilities.cpp | 2 +- src/android/camera_capabilities.h | 2 + src/android/camera_device.cpp | 702 ++++++++++++------ src/android/camera_device.h | 30 +- src/android/camera_request.cpp | 79 +- src/android/camera_request.h | 96 ++- src/android/camera_stream.cpp | 17 +- src/android/camera_stream.h | 10 +- src/android/jpeg/post_processor_jpeg.cpp | 17 +- src/android/jpeg/post_processor_jpeg.h | 2 +- src/android/post_processor.h | 4 +- src/android/yuv/post_processor_yuv.cpp | 2 +- src/android/yuv/post_processor_yuv.h | 2 +- src/libcamera/camera.cpp | 78 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 46 +- src/libcamera/pipeline_handler.cpp | 120 ++- src/libcamera/request.cpp | 159 ++++ 21 files changed, 1048 insertions(+), 377 deletions(-)