From patchwork Tue Dec 10 14:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22271 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C2996C3257 for ; Tue, 10 Dec 2024 14:26:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D37B67E92; Tue, 10 Dec 2024 15:26:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="fjuObykP"; dkim-atps=neutral Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40ACB67E8D for ; Tue, 10 Dec 2024 15:26:08 +0100 (CET) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2efded08c79so458098a91.0 for ; Tue, 10 Dec 2024 06:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840766; x=1734445566; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MbONCrxxlu5qaHg94AwjcNNxPrpZW9pFEh4Xyevk5rs=; b=fjuObykPNGo5tl8PKXzoumtT2XA53zsP7WEa8okWSOxbpkS/bmhew/3KR5wMBIvlAl mAh/610toZnb21r2xNcviqs+ghTJJzBVEob4yqq2mFzhjbriubb8YaNEgUZFBf0CGnzn DPni0673xV/RZ0+l4AjHc1pvxfcsFPZ+dX698= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840766; x=1734445566; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MbONCrxxlu5qaHg94AwjcNNxPrpZW9pFEh4Xyevk5rs=; b=RBG87gSyKWw5ScOypBh/aWTYG3xv2g8o8pxjkGQfR7fOp/EFxx9hRFni7vrxr7pCn4 l09Qj8LqjaGoG+/BKm7N3sb6FDTq6pRqR6qFW6NxQqO7iMc7T3glEl876tfVTBDSN3gT jN5Rnx6ivkmmQ5uQ+vPAFes9Re1Oj/XFUOaikL73pM+mIC3jK01vCQrS1n6sVxWXxo6h 4ukEKg8KUB5ddt39HB4gxN5jxXAYJ+2/6mxHOfpGKrqM/9re/h6zzwS8m4s9Hd9IyxWU iYtQNxBxWFkhEY42wKsSmrlLqQTtvPKV65KcpQQYsh/Q/lwBNhqy1NV1mtXVT9q7AiiD remg== X-Gm-Message-State: AOJu0YwZYHGbSjQB0P8erpsHXDcGTSyAZ5Fv7sdTLTc2H6Czqndq4mZx w7eyNhcYtwT1buc2ajaSY36dvo9MOigcR6BXx5/2oV1tqpTWDePz1fmdrhurbP1dLlxgH3sNbiM = X-Gm-Gg: ASbGnctt9aP9p8jRTHYBNS5RWg+sVO8K6v3sLXtPISwHRpjSivgq55l59hiHixd0PDa c56cN5gFvQu1kECbLQToqT3JWRJXX7grzFr7nH0jRJdQ0qplacw0uaMZ9BvRidJIQ4+/EO5dISV LCCMGM7O5x9A4R+0XwwEL1ZwqBMCt5rmJ6X8Px0N0O+cZpV6kEK1/wY+2dLVIkpKS91wZ28F+ka 2yroRsFo84Ghy5RKBktGh8+AgHizbcaHcWbHC02WVszXvh6TCf8VEJvgpRr6UexZ/8hzFmjsbU1 ulJPQQ98du0UVPprHZS+nuYOSn8kZxCymB6sVOVKWdnFzrbsU1WWvsHrqFZb X-Google-Smtp-Source: AGHT+IEUl5BISdsGpljYIFVeCZOVUu1qDBp8MgWvWI2WyWvnTSuR+FJcpBy2njOQkE2RA71z5lXuiw== X-Received: by 2002:a17:90b:5102:b0:2ef:e0bb:1ef2 with SMTP id 98e67ed59e1d1-2efe0bb1ff5mr2676540a91.19.1733840766405; Tue, 10 Dec 2024 06:26:06 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:06 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v4 3/7] android: Correctly support multiple Mapped streams Date: Tue, 10 Dec 2024 14:23:56 +0000 Message-ID: <20241210142557.2886315-4-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Android camera HAL supports creating streams for the Android framework by post-processing streams produced by libcamera. However at the moment a single Mapped stream can be associated with a Direct stream because, after the first post-processing, the data from the internal stream are returned preventing further post-processing passes. Fix this by storing the list of Internal stream buffers used as the post-processing source in an Camera3RequestDescriptor::internalBuffers_ map to replace the single internalBuffer_ pointer and return the internal buffers when the capture request is deleted. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 65 +++++++++++++++++++--------------- src/android/camera_request.cpp | 11 +++--- src/android/camera_request.h | 3 +- 3 files changed, 46 insertions(+), 33 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f6dadaf22..6bc19a74a 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -966,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * to a libcamera stream. Streams of type Mapped will be handled later. * * Collect the CameraStream associated to each requested capture stream. - * Since requestedStreams is an std:map<>, no duplications can happen. + * Since directBuffers is an std:map<>, no duplications can happen. */ - std::map requestedBuffers; + std::map directBuffers; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -1007,6 +1007,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques cameraStream->configuration().size); frameBuffer = buffer.frameBuffer.get(); acquireFence = std::move(buffer.fence); + + directBuffers[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -1015,13 +1017,17 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * Get the frame buffer from the CameraStream internal * buffer pool. * - * The buffer has to be returned to the CameraStream - * once it has been processed. + * The buffer will be returned to the CameraStream when + * the request is destroyed. */ frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; buffer.srcBuffer = frameBuffer; + /* + * Track the allocated internal buffers, which will be + * recycled when the descriptor is destroyed. + */ + descriptor->internalBuffers_[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (internal)"; descriptor->pendingStreamsToProcess_.insert( @@ -1037,8 +1043,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques auto fence = std::make_unique(std::move(acquireFence)); descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); - - requestedBuffers[cameraStream] = frameBuffer; } /* @@ -1076,24 +1080,43 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * post-processing. No need to recycle the buffer since it's * owned by Android. */ - auto iterDirectBuffer = requestedBuffers.find(sourceStream); - if (iterDirectBuffer != requestedBuffers.end()) { + auto iterDirectBuffer = directBuffers.find(sourceStream); + if (iterDirectBuffer != directBuffers.end()) { buffer.srcBuffer = iterDirectBuffer->second; continue; } /* - * If that's not the case, we need to add a buffer to the request - * for this stream. + * If that's not the case, we use an internal buffer allocated + * from the source stream. + */ + + /* + * If an internal buffer has been requested for the source + * stream before, we should reuse it. */ - FrameBuffer *frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; + auto iterInternalBuffer = descriptor->internalBuffers_.find(sourceStream); + if (iterInternalBuffer != descriptor->internalBuffers_.end()) { + buffer.srcBuffer = iterInternalBuffer->second; + continue; + } + + /* + * Otherwise, we need to create an internal buffer to the + * request for the source stream. Get the frame buffer from the + * source stream's internal buffer pool. + * + * The buffer will be returned to the CameraStream when the + * request is destructed. + */ + FrameBuffer *frameBuffer = sourceStream->getBuffer(); buffer.srcBuffer = frameBuffer; descriptor->request_->addBuffer(sourceStream->stream(), frameBuffer, nullptr); - requestedBuffers[sourceStream] = frameBuffer; + /* Track the allocated internal buffer. */ + descriptor->internalBuffers_[sourceStream] = frameBuffer; } /* @@ -1253,13 +1276,6 @@ void CameraDevice::requestComplete(Request *request) if (ret) { setBufferStatus(*buffer, StreamBuffer::Status::Error); descriptor->pendingStreamsToProcess_.erase(stream); - - /* - * If the framebuffer is internal to CameraStream return - * it back now that we're done processing it. - */ - if (buffer->internalBuffer) - stream->putBuffer(buffer->internalBuffer); } } @@ -1378,13 +1394,6 @@ void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, { setBufferStatus(*streamBuffer, status); - /* - * If the framebuffer is internal to CameraStream return it back now - * that we're done processing it. - */ - if (streamBuffer->internalBuffer) - streamBuffer->stream->putBuffer(streamBuffer->internalBuffer); - Camera3RequestDescriptor *request = streamBuffer->request; { diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 52a3ac1f7..92e74ab6a 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -10,6 +10,7 @@ #include #include "camera_buffer.h" +#include "camera_stream.h" using namespace libcamera; @@ -138,7 +139,12 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( request_ = camera->createRequest(reinterpret_cast(this)); } -Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; +Camera3RequestDescriptor::~Camera3RequestDescriptor() +{ + /* Recycle the allocated internal buffer back to its source stream. */ + for (auto &[sourceStream, frameBuffer] : internalBuffers_) + sourceStream->putBuffer(frameBuffer); +} /** * \class StreamBuffer @@ -166,9 +172,6 @@ Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; * \var StreamBuffer::status * \brief Track the status of the buffer * - * \var StreamBuffer::internalBuffer - * \brief Pointer to a buffer internally handled by CameraStream (if any) - * * \var StreamBuffer::srcBuffer * \brief Pointer to the source frame buffer used for post-processing * diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 335f1985d..6b2a00795 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -49,7 +49,6 @@ public: std::unique_ptr frameBuffer; libcamera::UniqueFD fence; Status status = Status::Success; - libcamera::FrameBuffer *internalBuffer = nullptr; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; Camera3RequestDescriptor *request; @@ -85,6 +84,8 @@ public: std::unique_ptr request_; std::unique_ptr resultMetadata_; + std::map internalBuffers_; + bool complete_ = false; Status status_ = Status::Success;