From patchwork Wed Dec 4 16:36:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22163 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87E51BDB1C for ; Wed, 4 Dec 2024 16:41:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 63852660DA; Wed, 4 Dec 2024 17:41:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="kGO1gp++"; dkim-atps=neutral Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C3FBB660CE for ; Wed, 4 Dec 2024 17:41:46 +0100 (CET) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-724f41d520cso18198b3a.1 for ; Wed, 04 Dec 2024 08:41:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330503; x=1733935303; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VzdjXSoSlpxLRIYtNCwI2M+hp0CGvhy+AVndgobM5W0=; b=kGO1gp++e9x0wz2vwzy8woXBeCHOdU9quydZyeWO2LEQWCnqjNc1ZzkB6+ywEXAIbp vpMd7pCzsCYJahn/3SORvBT7DtkVBP1HP0C2ojAEEVFtIQsjUpYztFaJckhCphu8gR56 gajBxmJXifGWEH+1x6J9/azBWw9iZxe1sSQdc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330503; x=1733935303; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VzdjXSoSlpxLRIYtNCwI2M+hp0CGvhy+AVndgobM5W0=; b=AVh38dkshE0RkWVmeezjzJrzBd9iE3F2X7KbXorbgggThBHvJMP2FjcdO8rh4KvxNJ TX0j5ZEGL6tHPr+RkTL6rYAiVqbr+pFduwx4KrMb03n8r4968nnVGR28qr33pD91A/2O pZoek+2JpCFmEVLSqgS2HMhU/bexjG2l9HamTpkz1lkLWquLw9hgyT8W3F31+NIjjLx1 EXgvje2wZNoqjCf/1RKO2dYMByeUl9Ck2Mg3N47nbMZmjjCR91fYO/VnsRQ55qGjZmln 5xZF4VEz9sdw0iEyNUhEmKiZT5BrEOWYo6urThImgErT9oZl/ca2EgR6aWexpd+aqltT kMWg== X-Gm-Message-State: AOJu0Yxjb3T/hdttxKYh+nchWF/4/2/2RszAQCZXgTKkg+TBISbDRQmm aTocsLwG1ySAmwfo3DUA6GbSSc+a3wZLk07xrKcn9fuC7mfGGDhPhobwXitbErMd+q9JG7+42w8 = X-Gm-Gg: ASbGncuMJzvKXDnqT10KYc2lRTtd/6fhwieTInrn4ej+t8V8MmBXkiX5J06St+uYxgO y8vbR5QWIqqX2Pz/M3g8rxER2ejM+jNq4gcVWtejR1YFd87JzMbFV8xFSAHwxx8tcxA6bjtsING zAnXLQGbhEMjyMnrejBrBNAnT8mCE9f64CdWMNr7VkuabVWw/Os6t/BWAOJjAtuEp5wRyOtR/c4 XOCNm1TgSBIPZHynaclo0MtewWtXMFIGLdacPQ/pOntfR/V6ecK83gyjrkkhgGdu6KSlqxsGGzp D/ZDd3lY+0mb7H+8uM5L7IdqUJPeN0ht72WtU/fXaVlgVzP58ql18vlU X-Google-Smtp-Source: AGHT+IGo45GOB+Fm7QiQsYAu9EZofbaNWuy9lwJ4pHns9VHYeuKt54oW6oZyeMIp6gYYUvKYTTL8Hw== X-Received: by 2002:a05:6a00:2e87:b0:725:9ac3:f22 with SMTP id d2e1a72fcca58-7259ac31270mr1191673b3a.1.1733330503326; Wed, 04 Dec 2024 08:41:43 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:42 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v3 1/7] android: Move StreamBuffer out of Camera3RequestDescriptor Date: Wed, 4 Dec 2024 16:36:26 +0000 Message-ID: <20241204164137.3938891-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move StreamBuffer out of Camera3RequestDescriptor before applying partial result to Android adaptor. This change makes it easier to add Camera3ResultDescriptor, which references both StreamBuffer and Camera3RequestDescriptor Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 22 +++++----- src/android/camera_device.h | 8 ++-- src/android/camera_request.cpp | 29 +++++++------ src/android/camera_request.h | 53 ++++++++++++++---------- src/android/camera_stream.cpp | 16 +++---- src/android/camera_stream.h | 6 +-- src/android/jpeg/encoder.h | 2 +- src/android/jpeg/encoder_jea.cpp | 2 +- src/android/jpeg/encoder_jea.h | 2 +- src/android/jpeg/encoder_libjpeg.cpp | 2 +- src/android/jpeg/encoder_libjpeg.h | 2 +- src/android/jpeg/post_processor_jpeg.cpp | 2 +- src/android/jpeg/post_processor_jpeg.h | 2 +- src/android/post_processor.h | 4 +- src/android/yuv/post_processor_yuv.cpp | 2 +- src/android/yuv/post_processor_yuv.h | 2 +- 16 files changed, 82 insertions(+), 74 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a038131ae..4e3bdc9cc 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -865,7 +865,7 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); for (auto &buffer : descriptor->buffers_) - buffer.status = Camera3RequestDescriptor::Status::Error; + buffer.status = StreamBuffer::Status::Error; descriptor->status_ = Camera3RequestDescriptor::Status::Error; } @@ -1163,7 +1163,7 @@ void CameraDevice::requestComplete(Request *request) if (fence) buffer.fence = fence->release(); } - buffer.status = Camera3RequestDescriptor::Status::Success; + buffer.status = StreamBuffer::Status::Success; } /* @@ -1226,12 +1226,12 @@ void CameraDevice::requestComplete(Request *request) auto iter = descriptor->pendingStreamsToProcess_.begin(); while (iter != descriptor->pendingStreamsToProcess_.end()) { CameraStream *stream = iter->first; - Camera3RequestDescriptor::StreamBuffer *buffer = iter->second; + StreamBuffer *buffer = iter->second; FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; - setBufferStatus(*buffer, Camera3RequestDescriptor::Status::Error); + setBufferStatus(*buffer, StreamBuffer::Status::Error); iter = descriptor->pendingStreamsToProcess_.erase(iter); continue; } @@ -1241,7 +1241,7 @@ void CameraDevice::requestComplete(Request *request) ++iter; int ret = stream->process(buffer); if (ret) { - setBufferStatus(*buffer, Camera3RequestDescriptor::Status::Error); + setBufferStatus(*buffer, StreamBuffer::Status::Error); descriptor->pendingStreamsToProcess_.erase(stream); /* @@ -1311,7 +1311,7 @@ void CameraDevice::sendCaptureResults() for (auto &buffer : descriptor->buffers_) { camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; - if (buffer.status == Camera3RequestDescriptor::Status::Success) + if (buffer.status == StreamBuffer::Status::Success) status = CAMERA3_BUFFER_STATUS_OK; /* @@ -1335,11 +1335,11 @@ void CameraDevice::sendCaptureResults() } } -void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &streamBuffer, - Camera3RequestDescriptor::Status status) +void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, + StreamBuffer::Status status) { streamBuffer.status = status; - if (status != Camera3RequestDescriptor::Status::Success) { + if (status != StreamBuffer::Status::Success) { notifyError(streamBuffer.request->frameNumber_, streamBuffer.stream->camera3Stream(), CAMERA3_MSG_ERROR_BUFFER); @@ -1363,8 +1363,8 @@ void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &strea * be generated from post-processing have been completed. Mark the descriptor as * complete using completeDescriptor() in that case. */ -void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *streamBuffer, - Camera3RequestDescriptor::Status status) +void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, + StreamBuffer::Status status) { setBufferStatus(*streamBuffer, status); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 194ca3030..c92ee1aa4 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -65,8 +65,8 @@ public: int configureStreams(camera3_stream_configuration_t *stream_list); int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); - void streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *bufferStream, - Camera3RequestDescriptor::Status status); + void streamProcessingComplete(StreamBuffer *bufferStream, + StreamBuffer::Status status); protected: std::string logPrefix() const override; @@ -97,8 +97,8 @@ private: void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); - void setBufferStatus(Camera3RequestDescriptor::StreamBuffer &buffer, - Camera3RequestDescriptor::Status status); + void setBufferStatus(StreamBuffer &buffer, + StreamBuffer::Status status); std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 0d45960d9..52a3ac1f7 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -141,7 +141,7 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; /** - * \struct Camera3RequestDescriptor::StreamBuffer + * \class StreamBuffer * \brief Group information for per-stream buffer of Camera3RequestDescriptor * * A capture request placed to the libcamera HAL can contain multiple streams. @@ -150,35 +150,35 @@ Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; * generation. The generation of the stream will depend on its type (refer to * the CameraStream::Type documentation). * - * \var Camera3RequestDescriptor::StreamBuffer::stream + * \var StreamBuffer::stream * \brief Pointer to the corresponding CameraStream * - * \var Camera3RequestDescriptor::StreamBuffer::camera3Buffer + * \var StreamBuffer::camera3Buffer * \brief Native handle to the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::frameBuffer + * \var StreamBuffer::frameBuffer * \brief Encapsulate the dmabuf handle inside a libcamera::FrameBuffer for * direct streams * - * \var Camera3RequestDescriptor::StreamBuffer::fence + * \var StreamBuffer::fence * \brief Acquire fence of the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::status + * \var StreamBuffer::status * \brief Track the status of the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::internalBuffer + * \var StreamBuffer::internalBuffer * \brief Pointer to a buffer internally handled by CameraStream (if any) * - * \var Camera3RequestDescriptor::StreamBuffer::srcBuffer + * \var StreamBuffer::srcBuffer * \brief Pointer to the source frame buffer used for post-processing * - * \var Camera3RequestDescriptor::StreamBuffer::dstBuffer + * \var StreamBuffer::dstBuffer * \brief Pointer to the destination frame buffer used for post-processing * - * \var Camera3RequestDescriptor::StreamBuffer::request + * \var StreamBuffer::request * \brief Back pointer to the Camera3RequestDescriptor to which the StreamBuffer belongs */ -Camera3RequestDescriptor::StreamBuffer::StreamBuffer( +StreamBuffer::StreamBuffer( CameraStream *cameraStream, const camera3_stream_buffer_t &buffer, Camera3RequestDescriptor *requestDescriptor) : stream(cameraStream), camera3Buffer(buffer.buffer), @@ -186,9 +186,8 @@ Camera3RequestDescriptor::StreamBuffer::StreamBuffer( { } -Camera3RequestDescriptor::StreamBuffer::~StreamBuffer() = default; +StreamBuffer::~StreamBuffer() = default; -Camera3RequestDescriptor::StreamBuffer::StreamBuffer(StreamBuffer &&) = default; +StreamBuffer::StreamBuffer(StreamBuffer &&) = default; -Camera3RequestDescriptor::StreamBuffer & -Camera3RequestDescriptor::StreamBuffer::operator=(Camera3RequestDescriptor::StreamBuffer &&) = default; +StreamBuffer &StreamBuffer::operator=(StreamBuffer &&) = default; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 5b479180f..335f1985d 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -26,7 +26,9 @@ class CameraBuffer; class CameraStream; -class Camera3RequestDescriptor +class Camera3RequestDescriptor; + +class StreamBuffer { public: enum class Status { @@ -34,27 +36,34 @@ public: Error, }; - struct StreamBuffer { - StreamBuffer(CameraStream *stream, - const camera3_stream_buffer_t &buffer, - Camera3RequestDescriptor *request); - ~StreamBuffer(); - - StreamBuffer(StreamBuffer &&); - StreamBuffer &operator=(StreamBuffer &&); - - CameraStream *stream; - buffer_handle_t *camera3Buffer; - std::unique_ptr frameBuffer; - libcamera::UniqueFD fence; - Status status = Status::Success; - libcamera::FrameBuffer *internalBuffer = nullptr; - const libcamera::FrameBuffer *srcBuffer = nullptr; - std::unique_ptr dstBuffer; - Camera3RequestDescriptor *request; - - private: - LIBCAMERA_DISABLE_COPY(StreamBuffer) + StreamBuffer(CameraStream *stream, + const camera3_stream_buffer_t &buffer, + Camera3RequestDescriptor *request); + ~StreamBuffer(); + + StreamBuffer(StreamBuffer &&); + StreamBuffer &operator=(StreamBuffer &&); + + CameraStream *stream; + buffer_handle_t *camera3Buffer; + std::unique_ptr frameBuffer; + libcamera::UniqueFD fence; + Status status = Status::Success; + libcamera::FrameBuffer *internalBuffer = nullptr; + const libcamera::FrameBuffer *srcBuffer = nullptr; + std::unique_ptr dstBuffer; + Camera3RequestDescriptor *request; + +private: + LIBCAMERA_DISABLE_COPY(StreamBuffer) +}; + +class Camera3RequestDescriptor +{ +public: + enum class Status { + Success, + Error, }; /* Keeps track of streams requiring post-processing. */ diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 1d68540d7..53f292d4b 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -112,14 +112,14 @@ int CameraStream::configure() worker_ = std::make_unique(postProcessor_.get()); postProcessor_->processComplete.connect( - this, [&](Camera3RequestDescriptor::StreamBuffer *streamBuffer, + this, [&](StreamBuffer *streamBuffer, PostProcessor::Status status) { - Camera3RequestDescriptor::Status bufferStatus; + StreamBuffer::Status bufferStatus; if (status == PostProcessor::Status::Success) - bufferStatus = Camera3RequestDescriptor::Status::Success; + bufferStatus = StreamBuffer::Status::Success; else - bufferStatus = Camera3RequestDescriptor::Status::Error; + bufferStatus = StreamBuffer::Status::Error; cameraDevice_->streamProcessingComplete(streamBuffer, bufferStatus); @@ -165,7 +165,7 @@ int CameraStream::waitFence(int fence) return -errno; } -int CameraStream::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +int CameraStream::process(StreamBuffer *streamBuffer) { ASSERT(type_ != Type::Direct); @@ -283,7 +283,7 @@ void CameraStream::PostProcessorWorker::start() Thread::start(); } -void CameraStream::PostProcessorWorker::queueRequest(Camera3RequestDescriptor::StreamBuffer *dest) +void CameraStream::PostProcessorWorker::queueRequest(StreamBuffer *dest) { { MutexLocker lock(mutex_); @@ -306,7 +306,7 @@ void CameraStream::PostProcessorWorker::run() if (state_ != State::Running) break; - Camera3RequestDescriptor::StreamBuffer *streamBuffer = requests_.front(); + StreamBuffer *streamBuffer = requests_.front(); requests_.pop(); locker.unlock(); @@ -316,7 +316,7 @@ void CameraStream::PostProcessorWorker::run() } if (state_ == State::Flushing) { - std::queue requests = + std::queue requests = std::move(requests_); locker.unlock(); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 395552dac..30f64f690 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -127,7 +127,7 @@ public: CameraStream *sourceStream() const { return sourceStream_; } int configure(); - int process(Camera3RequestDescriptor::StreamBuffer *streamBuffer); + int process(StreamBuffer *streamBuffer); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); void flush(); @@ -146,7 +146,7 @@ private: ~PostProcessorWorker(); void start(); - void queueRequest(Camera3RequestDescriptor::StreamBuffer *request); + void queueRequest(StreamBuffer *request); void flush(); protected: @@ -158,7 +158,7 @@ private: libcamera::Mutex mutex_; libcamera::ConditionVariable cv_; - std::queue requests_ + std::queue requests_ LIBCAMERA_TSA_GUARDED_BY(mutex_); State state_ LIBCAMERA_TSA_GUARDED_BY(mutex_) = State::Stopped; diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h index ed033c191..25425d92c 100644 --- a/src/android/jpeg/encoder.h +++ b/src/android/jpeg/encoder.h @@ -20,7 +20,7 @@ public: virtual ~Encoder() = default; virtual int configure(const libcamera::StreamConfiguration &cfg) = 0; - virtual int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + virtual int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) = 0; }; diff --git a/src/android/jpeg/encoder_jea.cpp b/src/android/jpeg/encoder_jea.cpp index 25dc43173..b3ad04638 100644 --- a/src/android/jpeg/encoder_jea.cpp +++ b/src/android/jpeg/encoder_jea.cpp @@ -33,7 +33,7 @@ int EncoderJea::configure(const libcamera::StreamConfiguration &cfg) return 0; } -int EncoderJea::encode(Camera3RequestDescriptor::StreamBuffer *buffer, +int EncoderJea::encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) { diff --git a/src/android/jpeg/encoder_jea.h b/src/android/jpeg/encoder_jea.h index 91115d2e9..79265ebc6 100644 --- a/src/android/jpeg/encoder_jea.h +++ b/src/android/jpeg/encoder_jea.h @@ -20,7 +20,7 @@ public: ~EncoderJea(); int configure(const libcamera::StreamConfiguration &cfg) override; - int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) override; diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp index cb242b5ec..cb123576a 100644 --- a/src/android/jpeg/encoder_libjpeg.cpp +++ b/src/android/jpeg/encoder_libjpeg.cpp @@ -180,7 +180,7 @@ void EncoderLibJpeg::compressNV(const std::vector> &planes) } } -int EncoderLibJpeg::encode(Camera3RequestDescriptor::StreamBuffer *buffer, +int EncoderLibJpeg::encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) { diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h index 4ac85c22e..de04b17d1 100644 --- a/src/android/jpeg/encoder_libjpeg.h +++ b/src/android/jpeg/encoder_libjpeg.h @@ -22,7 +22,7 @@ public: ~EncoderLibJpeg(); int configure(const libcamera::StreamConfiguration &cfg) override; - int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) override; int encode(const std::vector> &planes, diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index 89b8a401e..f5a90785d 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -106,7 +106,7 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, } } -void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +void PostProcessorJpeg::process(StreamBuffer *streamBuffer) { ASSERT(encoder_); diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index 6fe214577..5421f233e 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -22,7 +22,7 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override; + void process(StreamBuffer *streamBuffer) override; private: void generateThumbnail(const libcamera::FrameBuffer &source, diff --git a/src/android/post_processor.h b/src/android/post_processor.h index b504a3796..f2e00d2b1 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -27,7 +27,7 @@ public: virtual int configure(const libcamera::StreamConfiguration &inCfg, const libcamera::StreamConfiguration &outCfg) = 0; - virtual void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) = 0; + virtual void process(StreamBuffer *streamBuffer) = 0; - libcamera::Signal processComplete; + libcamera::Signal processComplete; }; diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index c998807b0..b05623b23 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -49,7 +49,7 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, return 0; } -void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +void PostProcessorYuv::process(StreamBuffer *streamBuffer) { const FrameBuffer &source = *streamBuffer->srcBuffer; CameraBuffer *destination = streamBuffer->dstBuffer.get(); diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index ed7bb1fbe..459c6685e 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -18,7 +18,7 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override; + void process(StreamBuffer *streamBuffer) override; private: bool isValidBuffers(const libcamera::FrameBuffer &source, From patchwork Wed Dec 4 16:36:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22164 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C5173BDB1C for ; Wed, 4 Dec 2024 16:41:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65461660E4; Wed, 4 Dec 2024 17:41:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="EvWqnMN+"; dkim-atps=neutral Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E0246660C5 for ; Wed, 4 Dec 2024 17:41:47 +0100 (CET) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-724f0f6300aso15974b3a.2 for ; Wed, 04 Dec 2024 08:41:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330505; x=1733935305; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7RSUBsgMlBRQcCiQn/OQffu9bSxAh/8WGY8/nToqTB8=; b=EvWqnMN+SnMlQ87ZHiYzcISvtpseaHXF7X6nZDoj//PCHzg1l+wnBxs6Y9f8uUb0oR lfH3oXrmxmLsYVdHo9YFGvUJnYw6ZVPMS7nX0/Ri8krv7XcwjK01/S6NyJTUKLP0hcP4 6DxJtxpT5p4d0YRSFOyfFXbbRxo8QIyZq4gD4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330505; x=1733935305; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7RSUBsgMlBRQcCiQn/OQffu9bSxAh/8WGY8/nToqTB8=; b=kIoMdBExMYt83esVqFfK4lyZ3B2YrzNFxdfk1WPifxR9Rz9mSvBij/f8UveOhROclQ woagx547yeck9iYVZQlWJqAbOuRX8HMHwQ/HTduGVtDXNGJ0T58vOk2hOfQXaVR7MKJe ALIzMwThRPEAqab4HATvspnQ2/llkw7fbNjnBd1hhEonRf0Qb+duN3/L6FzBzYRzk3yy tRcFRZw7iB8nDHTFChF5aXovM8/x6oc54BUHfM/KQ3iCAQ9M/rSnv8WI8jzA5Y1nMYXD mm90CDXk68A23Ba7lPJW+eJcKrZ8EQqM+rqPwH2h2Mx/gQMJg+Ol2PbEYDXMZOvdDjL7 py5A== X-Gm-Message-State: AOJu0YyXJAsoyyf5fIiTHX6RLTDv+hkXfjficJ862voDZ+WrGdScODJH aKOc2BF/jYMd89zLNlzFZ3P2aqaLqZmOn5/2ql1by0chPEtuqFH/G8usxplIGBB+DIjVbKBhAZM = X-Gm-Gg: ASbGnctsqz/mPqtlJVeZCX60Dq/hC4xhCDTnmv+MpuqMUJdWbVwoASmqzeXZ81CMM38 +urPwjN9fFMaCk+K8PM2yVavZ7HuKcuIcVEAgJw1zEyh8wFchZdBuAmv7kTsa7Gu8150e06kwpJ D3JjwwWaCUKZ4rEF1VfeveMUUZgKIPpGrYGDSgQjwsjxqYhvnygQrVVDcCW3XpzlD+ImmHEOtEl a5UBcWxfcNmmo7M6XmE/63zG0NbwXZvWeYNzNEUDatGG7EMt2p5T6kTKPlMS+2UyieXQDMMhJ2b nzDUtmxef6l+rYE9arD3xMk2juZyp+75gNdJiKZxRCSLSqQUeefu7GIe X-Google-Smtp-Source: AGHT+IEOsYVPm81SCTxU5wBUXi3YRlYDKu3rvWq6Rzjvp9uKo7xHZjHfcJrSAshsJyzYMAFQHOmQMQ== X-Received: by 2002:a05:6a00:3a16:b0:724:eac3:576a with SMTP id d2e1a72fcca58-7257fcca19fmr9391185b3a.25.1733330504992; Wed, 04 Dec 2024 08:41:44 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:44 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v3 2/7] android: Set StreamBuffer::srcBuffer in CameraDevice::processCaptureRequest Date: Wed, 4 Dec 2024 16:36:27 +0000 Message-ID: <20241204164137.3938891-3-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamBuffer::srcBuffer was set right before being processed, while it was already determined when being constructed. This patch sets the value in CameraDevice::processCaptureRequest. `requestedStreams` is refactored from std::set to std::map to find the correct srcBuffer accordingly. The change also helps cleaning up the post process pipeline's dependency on `Camera3RequestDescriptor::pendingStreamsToProcess_`, which is going to be cleaned up in the following patches. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 4e3bdc9cc..f6dadaf22 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -9,7 +9,6 @@ #include #include -#include #include #include #include @@ -967,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * to a libcamera stream. Streams of type Mapped will be handled later. * * Collect the CameraStream associated to each requested capture stream. - * Since requestedStreams is an std:set<>, no duplications can happen. + * Since requestedStreams is an std:map<>, no duplications can happen. */ - std::set requestedStreams; + std::map requestedBuffers; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -1021,6 +1020,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques */ frameBuffer = cameraStream->getBuffer(); buffer.internalBuffer = frameBuffer; + buffer.srcBuffer = frameBuffer; + LOG(HAL, Debug) << ss.str() << " (internal)"; descriptor->pendingStreamsToProcess_.insert( @@ -1037,7 +1038,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); - requestedStreams.insert(cameraStream); + requestedBuffers[cameraStream] = frameBuffer; } /* @@ -1067,9 +1068,19 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * added to the request. */ CameraStream *sourceStream = cameraStream->sourceStream(); - ASSERT(sourceStream); - if (requestedStreams.find(sourceStream) != requestedStreams.end()) + ASSERT(sourceStream && sourceStream->type() == CameraStream::Type::Direct); + + /* + * If the buffer for the source stream has been requested as + * Direct, use its framebuffer as the source buffer for + * post-processing. No need to recycle the buffer since it's + * owned by Android. + */ + auto iterDirectBuffer = requestedBuffers.find(sourceStream); + if (iterDirectBuffer != requestedBuffers.end()) { + buffer.srcBuffer = iterDirectBuffer->second; continue; + } /* * If that's not the case, we need to add a buffer to the request @@ -1077,11 +1088,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques */ FrameBuffer *frameBuffer = cameraStream->getBuffer(); buffer.internalBuffer = frameBuffer; + buffer.srcBuffer = frameBuffer; descriptor->request_->addBuffer(sourceStream->stream(), frameBuffer, nullptr); - requestedStreams.insert(sourceStream); + requestedBuffers[sourceStream] = frameBuffer; } /* @@ -1236,8 +1248,6 @@ void CameraDevice::requestComplete(Request *request) continue; } - buffer->srcBuffer = src; - ++iter; int ret = stream->process(buffer); if (ret) { From patchwork Wed Dec 4 16:36:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22165 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A76CBDB1C for ; Wed, 4 Dec 2024 16:41:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B7664660EB; Wed, 4 Dec 2024 17:41:55 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="NO/CFyNm"; dkim-atps=neutral Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9AD12660DB for ; Wed, 4 Dec 2024 17:41:49 +0100 (CET) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-7251abe0e69so38862b3a.0 for ; Wed, 04 Dec 2024 08:41:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330506; x=1733935306; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=grGNlOBle1EviC0OZSGgVRLbocaEWZG8kJhPP01rB0g=; b=NO/CFyNmAGOkfksYLx3K8Z3cPN4qeb6Z722losMfJnSYgEU4NZgPh1DUBtRTg2LUVL /3OlPnO+sQPsTpJI1u14dt0YpMQVb8X2187QQCHaCRORfWYR/4/oOwXwCiQI9PFRFB7s rqDcczHlSdbzHyQlmq0k/NneAIgm/uv6hnDl4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330506; x=1733935306; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=grGNlOBle1EviC0OZSGgVRLbocaEWZG8kJhPP01rB0g=; b=VQca6SumaL+egH5x+TDRRP52v1NAHJWPH4GpCpcNDDFjGaElG+uugI0j8R2+HfABxC bfs3sP9qGidAXZu7V6VOR3MDFiK5/MsBrZqa9RxNH1DxqOKeIZuoD9Ox+0hX7Z13/4Kl VFIZNgaTBXKfUZgWcroPan+R32aEgdHoUDAvqTlXjVO5b3LNbkdHlNu9Wq+ABLyxXkME PuNlGJrQeSpqlgB1+22pxfpFvGGjkf9b5mpEMqEzO6sT7Q7556eDP00wKYxCstnqy3FZ 3QULjJfFC9cfD3bAMjdlMVs3V/1dlr1A5IWtq2vI8+nTixjzvJAabgd6QlnSNtRAWNlI 06PQ== X-Gm-Message-State: AOJu0YzDFHQdSL97FXIrVc+Q/J8QAFoTXoTtR40j2iKbQNxr7ja6+/hp m0WrsmctLDch3CF4OhrO7VIGYl/6qNJrjTBzUtvKkTzhFufSvOr6B0deUp5SfSzkxIDayUt7pvM = X-Gm-Gg: ASbGncvKPeX+lT9bMWMDev3TP/nu52gp3qDVn8JKls2wXjHN1uALT8cl0ISEl+dm4An WVdX2xSnE/it33a2wT1fHZUso2PgbPawEwncAHE3V9jj0fT4FqX6Inwg5fdteADNzBsB5EOxr2H iPkDS9t3kkd+aFkVOzK9oB/s8+gD74ziDCgTBOikoZv6mmqKJgGUsfaj82uVT5dIvvzEcz4b4Op l1HzWsRSq/D9Noi40HLdGhZdBkXpgkaQRZCo/rihtG2brdYnxMCafGwc2q4U3kg/Xhb5kcq5/Rm VU1lwqyhbIog0OLVqk2WneHGiK+TGw+fRY8+I+//3QWK9Rvg8eaLKTdW X-Google-Smtp-Source: AGHT+IEm6dpeuC2o/gdURtr1wPQ/Dk/B5Ccav2NswUj7G2jJwJRHLVwtCcfuz91HhJYPPL5x3vukNw== X-Received: by 2002:a05:6a00:1703:b0:724:62b3:58da with SMTP id d2e1a72fcca58-7257fa59b65mr8875202b3a.6.1733330506528; Wed, 04 Dec 2024 08:41:46 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:46 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v3 3/7] android: Correctly support multiple Mapped streams Date: Wed, 4 Dec 2024 16:36:28 +0000 Message-ID: <20241204164137.3938891-4-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Android camera HAL supports creating streams for the Android framework by post-processing streams produced by libcamera. However at the moment a single Mapped stream can be associated with a Direct stream because, after the first post-processing, the data from the internal stream are returned preventing further post-processing passes. Fix this by storing the list of Direct stream buffers used as the post-processing source in an Camera3RequestDescriptor::internalBuffers_ map to replace the single internalBuffer_ pointer and return the internal buffers when the capture request is deleted. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 66 +++++++++++++++++++--------------- src/android/camera_request.cpp | 11 +++--- src/android/camera_request.h | 3 +- 3 files changed, 47 insertions(+), 33 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f6dadaf22..f2dd8d4fd 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -966,9 +966,10 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * to a libcamera stream. Streams of type Mapped will be handled later. * * Collect the CameraStream associated to each requested capture stream. - * Since requestedStreams is an std:map<>, no duplications can happen. + * Since directBuffers is an std:map<>, no duplications can + * happen. */ - std::map requestedBuffers; + std::map directBuffers; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -1007,6 +1008,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques cameraStream->configuration().size); frameBuffer = buffer.frameBuffer.get(); acquireFence = std::move(buffer.fence); + + directBuffers[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -1015,13 +1018,17 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * Get the frame buffer from the CameraStream internal * buffer pool. * - * The buffer has to be returned to the CameraStream - * once it has been processed. + * The buffer will be returned to the CameraStream when + * the request is destroyed. */ frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; buffer.srcBuffer = frameBuffer; + /* + * Track the allocated internal buffers, which will be + * recycled when the descriptor destroyed. + */ + descriptor->internalBuffers_[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (internal)"; descriptor->pendingStreamsToProcess_.insert( @@ -1037,8 +1044,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques auto fence = std::make_unique(std::move(acquireFence)); descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); - - requestedBuffers[cameraStream] = frameBuffer; } /* @@ -1076,24 +1081,43 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * post-processing. No need to recycle the buffer since it's * owned by Android. */ - auto iterDirectBuffer = requestedBuffers.find(sourceStream); - if (iterDirectBuffer != requestedBuffers.end()) { + auto iterDirectBuffer = directBuffers.find(sourceStream); + if (iterDirectBuffer != directBuffers.end()) { buffer.srcBuffer = iterDirectBuffer->second; continue; } /* - * If that's not the case, we need to add a buffer to the request - * for this stream. + * If that's not the case, we use an internal buffer allocated + * from the source stream. + */ + + /* + * If an internal buffer has been requested for the source + * stream before, we should reuse it. */ - FrameBuffer *frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; + auto iterInternalBuffer = descriptor->internalBuffers_.find(sourceStream); + if (iterInternalBuffer != descriptor->internalBuffers_.end()) { + buffer.srcBuffer = iterInternalBuffer->second; + continue; + } + + /* + * Otherwise, we need to create an internal buffer to the + * request for the source stream. Get the frame buffer from the + * source stream's internal buffer pool. + * + * The buffer will be returned to the CameraStream when the + * request is destructed. + */ + FrameBuffer *frameBuffer = sourceStream->getBuffer(); buffer.srcBuffer = frameBuffer; descriptor->request_->addBuffer(sourceStream->stream(), frameBuffer, nullptr); - requestedBuffers[sourceStream] = frameBuffer; + /* Track the allocated internal buffer. */ + descriptor->internalBuffers_[sourceStream] = frameBuffer; } /* @@ -1253,13 +1277,6 @@ void CameraDevice::requestComplete(Request *request) if (ret) { setBufferStatus(*buffer, StreamBuffer::Status::Error); descriptor->pendingStreamsToProcess_.erase(stream); - - /* - * If the framebuffer is internal to CameraStream return - * it back now that we're done processing it. - */ - if (buffer->internalBuffer) - stream->putBuffer(buffer->internalBuffer); } } @@ -1378,13 +1395,6 @@ void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, { setBufferStatus(*streamBuffer, status); - /* - * If the framebuffer is internal to CameraStream return it back now - * that we're done processing it. - */ - if (streamBuffer->internalBuffer) - streamBuffer->stream->putBuffer(streamBuffer->internalBuffer); - Camera3RequestDescriptor *request = streamBuffer->request; { diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 52a3ac1f7..92e74ab6a 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -10,6 +10,7 @@ #include #include "camera_buffer.h" +#include "camera_stream.h" using namespace libcamera; @@ -138,7 +139,12 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( request_ = camera->createRequest(reinterpret_cast(this)); } -Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; +Camera3RequestDescriptor::~Camera3RequestDescriptor() +{ + /* Recycle the allocated internal buffer back to its source stream. */ + for (auto &[sourceStream, frameBuffer] : internalBuffers_) + sourceStream->putBuffer(frameBuffer); +} /** * \class StreamBuffer @@ -166,9 +172,6 @@ Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; * \var StreamBuffer::status * \brief Track the status of the buffer * - * \var StreamBuffer::internalBuffer - * \brief Pointer to a buffer internally handled by CameraStream (if any) - * * \var StreamBuffer::srcBuffer * \brief Pointer to the source frame buffer used for post-processing * diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 335f1985d..6b2a00795 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -49,7 +49,6 @@ public: std::unique_ptr frameBuffer; libcamera::UniqueFD fence; Status status = Status::Success; - libcamera::FrameBuffer *internalBuffer = nullptr; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; Camera3RequestDescriptor *request; @@ -85,6 +84,8 @@ public: std::unique_ptr request_; std::unique_ptr resultMetadata_; + std::map internalBuffers_; + bool complete_ = false; Status status_ = Status::Success; From patchwork Wed Dec 4 16:36:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22166 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 72964BDB1C for ; Wed, 4 Dec 2024 16:41:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01951660EE; Wed, 4 Dec 2024 17:41:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Hx02CMGh"; dkim-atps=neutral Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 350E2660C5 for ; Wed, 4 Dec 2024 17:41:51 +0100 (CET) Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-21572ca3cccso38664565ad.2 for ; Wed, 04 Dec 2024 08:41:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330508; x=1733935308; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SW0ReRfQT80HjLN0627g1Oa3EjdkMjs9/avml1BZiic=; b=Hx02CMGhIvUuKLDjp10qxPdPfEe3CwgWKlLZABIcK9n9UXW49Nek2kIi6+fYZSTX6K gyx90hqy97UQ/3RFJMe/6jCuduN2ownzLO2cHZM0HnZeKNbj1mfnZ5k+N2xKdOktr6Lu ttho2z8vsemG3OSC1GwjeAezJv3owMeMVfpMA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330508; x=1733935308; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SW0ReRfQT80HjLN0627g1Oa3EjdkMjs9/avml1BZiic=; b=BvnACHeqyR4Mfg5Rbi+8lQwNJ2U2BZItfWeynfOH+KFd/Gtg5r26sQ6aBJOmdQH7sR ego8+oAckq0mogRQ8HHvH36Sa2nMDXTngkBROgig0WUjEAB2bLrYKG+zBQKm8LAaKrtP iEDqmfnD+UY87HK7QGSpy3l64oRzEs0far6//HVUokhfuxhFer3g9krJDrtiCGSATENP 0LMOWhUCaVHgcP0zK9mL5YktZHeZ3SBVbcuFw3agxQ/CR8JWH2AdYLcEh+bTs8Y7wi+5 hKgNY8C/jf8RSrwttkb6ayel3lvEUaJXSHhat1IZAlswUlzChbUnX7w1LWXtQlSaisfc AwtQ== X-Gm-Message-State: AOJu0YzTyRR+sqDfsIwMmE+rv9cEMDtAcvAqcZoCGv9kzHKyLfZs3h0y JluEgtSFYi5XrM1Vt5KtQ807j28XCs4EXMjKjy4ZcsA5ZT08jF6iGOiwDjp4oc7i4T/4PXFv7t8 = X-Gm-Gg: ASbGncuL2Uox8Lno0NNviRbzKzGna6hdTzk92Arf417kqF3Do+pfTf97mTxno3rFzj/ 4FgeCQDWvHjwZrp5Bs9u7NMPcGIxEbFjFV00/OYBlWOnGNfBcVYoYyqil9E59HkpfHptLGuowfZ WGTTN3ceoHuSI9n2jMT5Yrj+80ytsS4csWY6gBnmOdTIwkhAe3d5brWsM01M9ukfSwrzM696axE Jb9nePikeDfceZwA8K47ZLEx2D8U9ECj0oRrqzzokc3/fEtbSGjPPIPsKbWABOqlH632Hxm0a+q 0GAFF45Z9BHkI9IgLlanevO1R3yS4UsQQ23IieE8KoGCLURAvOlbWIw9 X-Google-Smtp-Source: AGHT+IGtYS/y7Qc8dbU0w8S14phcpaspCB7oQFpIJJhQ1iSeKtXIAW+JGdNReK/9N8vGShN+d1UKQg== X-Received: by 2002:a17:903:1790:b0:215:711f:4979 with SMTP id d9443c01a7336-215bd24b939mr107924905ad.35.1733330508309; Wed, 04 Dec 2024 08:41:48 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:47 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v3 4/7] android: Add CameraDevice::sendCaptureResult() Date: Wed, 4 Dec 2024 16:36:29 +0000 Message-ID: <20241204164137.3938891-5-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The new function is separated from sendCaptureResults(), which allows other functions to send a result out of order, like aborting. It'll be updated in the upcoming patch, when the usage of error result is separated. This function also allows the upcoming partial results to be sent back to the application without waiting for the requests. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 57 ++++++++++++++++++----------------- src/android/camera_device.h | 1 + 2 files changed, 31 insertions(+), 27 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f2dd8d4fd..497d363d6 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1324,42 +1324,45 @@ void CameraDevice::sendCaptureResults() auto descriptor = std::move(descriptors_.front()); descriptors_.pop(); - camera3_capture_result_t captureResult = {}; + sendCaptureResult(descriptor.get()); + } +} - captureResult.frame_number = descriptor->frameNumber_; +void CameraDevice::sendCaptureResult(Camera3RequestDescriptor *request) const +{ + std::vector resultBuffers; + resultBuffers.reserve(request->buffers_.size()); - if (descriptor->resultMetadata_) - captureResult.result = - descriptor->resultMetadata_->getMetadata(); + for (auto &buffer : request->buffers_) { + camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; - std::vector resultBuffers; - resultBuffers.reserve(descriptor->buffers_.size()); + if (buffer.status == StreamBuffer::Status::Success) + status = CAMERA3_BUFFER_STATUS_OK; - for (auto &buffer : descriptor->buffers_) { - camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; + /* + * Pass the buffer fence back to the camera framework as + * a release fence. This instructs the framework to wait + * on the acquire fence in case we haven't done so + * ourselves for any reason. + */ + resultBuffers.push_back({ buffer.stream->camera3Stream(), + buffer.camera3Buffer, status, + -1, buffer.fence.release() }); + } - if (buffer.status == StreamBuffer::Status::Success) - status = CAMERA3_BUFFER_STATUS_OK; + camera3_capture_result_t captureResult = {}; - /* - * Pass the buffer fence back to the camera framework as - * a release fence. This instructs the framework to wait - * on the acquire fence in case we haven't done so - * ourselves for any reason. - */ - resultBuffers.push_back({ buffer.stream->camera3Stream(), - buffer.camera3Buffer, status, - -1, buffer.fence.release() }); - } + captureResult.frame_number = request->frameNumber_; + captureResult.num_output_buffers = resultBuffers.size(); + captureResult.output_buffers = resultBuffers.data(); - captureResult.num_output_buffers = resultBuffers.size(); - captureResult.output_buffers = resultBuffers.data(); + if (request->status_ == Camera3RequestDescriptor::Status::Success) + captureResult.partial_result = 1; - if (descriptor->status_ == Camera3RequestDescriptor::Status::Success) - captureResult.partial_result = 1; + if (request->resultMetadata_) + captureResult.result = request->resultMetadata_->getMetadata(); - callbacks_->process_capture_result(callbacks_, &captureResult); - } + callbacks_->process_capture_result(callbacks_, &captureResult); } void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index c92ee1aa4..699aa8f17 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -97,6 +97,7 @@ private: void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); + void sendCaptureResult(Camera3RequestDescriptor *request) const; void setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status); std::unique_ptr getResultMetadata( From patchwork Wed Dec 4 16:36:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22167 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2827EBDB1C for ; Wed, 4 Dec 2024 16:42:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E51A660F1; Wed, 4 Dec 2024 17:41:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="jmk5/kHa"; dkim-atps=neutral Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4FB62660CE for ; Wed, 4 Dec 2024 17:41:53 +0100 (CET) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-724f0f6300aso16057b3a.2 for ; Wed, 04 Dec 2024 08:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330510; x=1733935310; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CC96l7K+Y3IIaBzL/E1qpGzpDcR7HOUllb+44i9I0Z4=; b=jmk5/kHaFoBwiydYFw86uQDU404+4I0Zw/nmg8sJ4o6WEiTEATLRshe1tN2O0lyvHv waHvrEzHcwFuor49BpRqjWA9sUlAfujcEOrWCHnjCM7na3HXo0R12hyXdoubvYnMI9Y9 4GgCSkvnIsiO9uzN8HsKArywhcgoQFkOU2VUA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330510; x=1733935310; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CC96l7K+Y3IIaBzL/E1qpGzpDcR7HOUllb+44i9I0Z4=; b=PB4Q6KLcovE68Iwdjy8aC4UnHCbKTIauuqTBVmDalE1DmwZBkH3x86Z2pm3AUxPtrK dnhD8hWU/BBFPH437iktcwfY3jl4dhDqA+ykxI79nbznNzeqeHJeoL1DZmIJ01oTZPt8 pXHfZP+neAT3nj6RpKq0a2hmaDiUuBIzTrV33MF1iLL7k6B+7qwkeT87dUdcSVQs3YGJ LV3D6yKW7xopjqEUrhUnAD4VRfdAy5j0tw9ebSXk/jKLnqq8gmVJem6ZOtXPhG12jnZI YYKNX6PdQO5WpvAgsFsF7bI8j1dsyh68Iddd4dPHL0HKd8f3R47GicWc7B8xs/Kfyv/w 9wZA== X-Gm-Message-State: AOJu0YymKiiDI//Re7GjPT6LfOTKEqIWxAfEBR6jeUIrwXxqsqxC0fdE c3/OCc3ZMG/tIEQqFKkO4sfTLbz6bdOn+XFKPZjZWQtM9Uc++VuHnEni2CjVdyDnhykhJVc5Q+o = X-Gm-Gg: ASbGncu74jGGTgzR7LoPpjTi/EdR5e6OIHlAsGWdImpsSrkPTpWqbNc/wp6fZ8lZ8Vj hXUj3dg/TQEypVyaIxQ8XCF8GwTsSCQdxuY0JdZH7OGlzkjGuUeGtu2FyCERzvruLXtMwen3s9O FoEY+f8ZYw40OgDf0MMRFvA7DimYkkZmR5MKWnqkdbrw+mugDLUDQrBQGB4gZ8MnSMVemttVp6Z 2XDDN2fKqbQ2dXL0PhCjsLs9QBXt+gczNbYr9BiO0VQIDPqkG4vYs3YTRUd+yttK7P+e5h4/Wx4 eYb+p2fIPcDPoDfZ/FPjM60NCziaEJaj0mocjnr680pCDLp+fdTafaFs X-Google-Smtp-Source: AGHT+IGIsZxEytTC18u9StxsGrQ2lIg5ZzB2dcCeMsKtfpSKuid0qJ3mtNTT/K3AQXsOE5RVBXtA7A== X-Received: by 2002:a05:6a00:92a7:b0:725:996c:3a16 with SMTP id d2e1a72fcca58-725996c3e4cmr1836681b3a.13.1733330509824; Wed, 04 Dec 2024 08:41:49 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:49 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v3 5/7] android: Drop notify CAMERA3_MSG_ERROR_REQUEST when a request fails Date: Wed, 4 Dec 2024 16:36:30 +0000 Message-ID: <20241204164137.3938891-6-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" According to Android Camera API v3.2, CAMERA3_MSG_ERROR_REQUEST is used for requests that have not done any processing. When a request is completed with failure, CAMERA3_MSG_ERROR_RESULT should be used instead. To avoid code duplication, when CameraMetadata cannot be generated, CAMERA3_MSG_ERROR_RESULT is notified after process_capture_result. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 497d363d6..dd2c603e0 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1211,10 +1211,7 @@ void CameraDevice::requestComplete(Request *request) << " not successfully completed: " << request->status(); - abortRequest(descriptor); - completeDescriptor(descriptor); - - return; + descriptor->status_ = Camera3RequestDescriptor::Status::Error; } /* @@ -1239,7 +1236,7 @@ void CameraDevice::requestComplete(Request *request) */ descriptor->resultMetadata_ = getResultMetadata(*descriptor); if (!descriptor->resultMetadata_) { - notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_RESULT); + descriptor->status_ = Camera3RequestDescriptor::Status::Error; /* * The camera framework expects an empty metadata pack on error. @@ -1325,6 +1322,16 @@ void CameraDevice::sendCaptureResults() descriptors_.pop(); sendCaptureResult(descriptor.get()); + + /* + * Call notify with CAMERA3_MSG_ERROR_RESULT to indicate some + * of the expected result metadata might not be available + * because the capture is cancelled by the camera. Only notify + * it when the final result is sent, since Android will ignore + * the following metadata. + */ + if (descriptor->status_ == Camera3RequestDescriptor::Status::Error) + notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_RESULT); } } From patchwork Wed Dec 4 16:36:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22168 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BE70BDB1C for ; Wed, 4 Dec 2024 16:42:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C0B1660F3; Wed, 4 Dec 2024 17:41:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="RQ22hJ8v"; dkim-atps=neutral Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B8AA660E2 for ; Wed, 4 Dec 2024 17:41:54 +0100 (CET) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-7252fba4de1so42745b3a.0 for ; Wed, 04 Dec 2024 08:41:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330511; x=1733935311; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=37nnvTi9vvdECb0kx0poJ2dB18jSK5vuzdObv0ENTJg=; b=RQ22hJ8vZyl/QIdqKpu9waswyinhyXegJu0MNHysH2YHl7HlnvPlNRU74afFZCoiwm kBKgTH9ba5okxYrWbVGLRlNvV/DST12sJhDJBABwYuJVUIQKqt550ILr4m77fx6guE+3 jgbDsw8VsbWDx1Tu+1ve0dP9yqJVTdciTVNHI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330511; x=1733935311; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=37nnvTi9vvdECb0kx0poJ2dB18jSK5vuzdObv0ENTJg=; b=cC2w037/DjnSFSO9Hw3/B7QNL/2X7c8kMWABJUUX3d6QQB5k9WK3J5zsHyDAY4gSkZ q70GHgsBhu++JuwxpOQat5tNNrqScMDPFrcKjKJ3hHZRLwnmw2Ywn/cPWvK1qWAGjv9v 2OBTIzxeVhTjvVl3J+yLtQyJ7KPRYT9RS7cAoqPFzQYl1fXoDnzwgSEIwa1yn24SLo0i zrrPyu6a/zWHZXT0xzKjwuX2Er0PS3yefZsnJ6AXAHOyiLte/CETkKk2tOb43FoOBTY1 rbqf9q4/cRfLx969oANY6V+W34jJ16FUkkK4WiK5GYGjqdFYJyQ7v9gh0Kf8Af4UMCwH Ce3Q== X-Gm-Message-State: AOJu0YzMaxiiCRwxsgxyRvil5GBpLJ1QuX1EFq62YXE05t0ZWRcYDFCG uoeTrzcX7eNA+lXey+JCaP8x2MNuZLiuhwvXgkF1LGSXWUA8mgg+GgVoVtkPtkJWeHU6sUhr58s = X-Gm-Gg: ASbGncsxf0SeJRqGDuPw/vnGUX4XDtlHkoiBqK8BxrQw7tRfcr4qUt2UjsXfNMCgl7w SEVQUh8ZvJARwnFwEhOph/SpA2fdhnX/YHawQbsBGj8wKKZC9JPAyfq/mH5weGr071rpIPAd1Q8 02lfM+UKKRdZ7aQrofo3J/T0OY+SA1u3W10Sw9gIO5djDh11ENBqCS+Ava3C5Hp6lS2cVP+zfSy LjLA+IJdxzDswnRIfDvmmuQoIInLcYbvQbqO5h0agIj3EcKVqfyndIullJcgcJpvPhLWpeFWzyh pFWnxgHc83GixjLgQQcN4ZUTxA+EkybkTnnn76bUY9aeGAMC9TPW42TJ X-Google-Smtp-Source: AGHT+IH9exai7cVZ1xtRic5GGuqexbN4H4apGhto6REw8K4B0fPJBqZh/pytnN7YTlOz1pYIfd7eQw== X-Received: by 2002:a05:6a00:14d5:b0:71e:60fc:ad11 with SMTP id d2e1a72fcca58-7257fcb04ddmr10464905b3a.16.1733330511625; Wed, 04 Dec 2024 08:41:51 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:50 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v3 6/7] android: notify CAMERA3_MSG_ERROR_REQUEST out of order Date: Wed, 4 Dec 2024 16:36:31 +0000 Message-ID: <20241204164137.3938891-7-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request hasn't done any processing, CAMERA3_MSG_ERROR_REQUEST and the following process_capture_result don't need to wait for the previous requests' completion. This patch avoids pushing the aborted requests into the request queue. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index dd2c603e0..18e974f56 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -867,6 +867,8 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const buffer.status = StreamBuffer::Status::Error; descriptor->status_ = Camera3RequestDescriptor::Status::Error; + + sendCaptureResult(descriptor); } bool CameraDevice::isValidRequest(camera3_capture_request_t *camera3Request) const @@ -1136,14 +1138,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques MutexLocker stateLock(stateMutex_); if (state_ == State::Flushing) { - Camera3RequestDescriptor *rawDescriptor = descriptor.get(); - { - MutexLocker descriptorsLock(descriptorsMutex_); - descriptors_.push(std::move(descriptor)); - } - abortRequest(rawDescriptor); - completeDescriptor(rawDescriptor); - + abortRequest(descriptor.get()); return 0; } From patchwork Wed Dec 4 16:36:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22169 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B8A08BDB1C for ; Wed, 4 Dec 2024 16:42:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 053BE660ED; Wed, 4 Dec 2024 17:42:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="kBtH2XNC"; dkim-atps=neutral Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD043660EC for ; Wed, 4 Dec 2024 17:41:56 +0100 (CET) Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-7242f559a9fso30537b3a.1 for ; Wed, 04 Dec 2024 08:41:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733330513; x=1733935313; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LcsRfFPzzANl9rpdLlsmIw3KAWzqxumyatnPESv6yy0=; b=kBtH2XNCLyNCaqY2wmoLZwNKFtp8K/slmgxxQkziJdvgrH1OqccdVGcUTiCY0uVaOQ Lx5Jhbn2uoPjGUf7wv7jyRP61J43J6YyFqVlvK2ANQV8Kj30yWGKbkX9UXeTfPIkjKdY yrWr2oLU5v3uyMDegQw+Pht99WBFmgWxtJFeE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733330513; x=1733935313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LcsRfFPzzANl9rpdLlsmIw3KAWzqxumyatnPESv6yy0=; b=BlicmEQJ24fAW7vl1ZmPbrtISjj7fhIH5zp1gs5827YJ5d2ICf+WQmOYZsQcJT5UP7 KcFzBM90Gw+2v2UDGa8dVr3oE6kwUdxwWxRf1Ld91qbJk+6R1KkZA9/QJUfzADf2BIQl Fcr4hcDJhNgOj0N6iWQ5qhLWO3dTZjiOOds9KKuZRndzNPP5jQ4Xyyrfrovc7y0pf4D6 PKDiFGKomeZ/uEaA8KjFwVYOPHZarHHLi33Wk0Pg1nevUgXpKerfDZK/llusC5aEKwyH byScxgbRSglCfZJjq+NKLOIAxlrP4QhzwDWm80zS83ePh6MosvYLhr6/O3lNgNc2+CWw e1tg== X-Gm-Message-State: AOJu0YxdiK6Gbs/8kzEhcaLb7N2ATBCoVh2U9dHoxFsAjyqBml+4yF1R Rp/7gAiX9E32oAjymtTMj6SuoGslGYazuIgxgm2NSUsATSZKsK5DBHCX1GGOQZJy4yt9bHfnWjs = X-Gm-Gg: ASbGncv5OtQ6w2sXqGIqcYGishjgDY/QcZesQ+BbaGOX2oyqQJSkK7KRFKPQMe0oNbt WpPA39XXQ4Fc9MRrPlxE/C7jrVs3V0a+xG0oxqJS8oDFoSSVQ3ugc32bFDJrb3JgH6b8HlzNsBZ p1qxemIdAuXcl8SHi8vnF7D/IyXGCr+MAUiQVasACIvVq5ZpOq2PyE331oSp2UyL99+JjJHJm+j /YpgoNOlNUX1zJ5YpoZfai6RjkH6mlPgIVdSdxPRi70J1XJPmZy2rP8RnEI4GZEAby1ouEjdDIs 1X19YK9xt1p13sGY6a9LSQzsPyh/QDYLFcOx+K6n7E4vJppb+gD43f9a X-Google-Smtp-Source: AGHT+IHC4shiNXtPjF0zbg6RzFr3W5vWujFZzmYbm4Dm+KerLCezwoM9+4UWYWaj7y1k5fCXq1bDpA== X-Received: by 2002:a05:6a00:3a1b:b0:71e:2a0:b0d0 with SMTP id d2e1a72fcca58-7257fa7167emr9211118b3a.13.1733330513509; Wed, 04 Dec 2024 08:41:53 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417725e9sm12551779b3a.80.2024.12.04.08.41.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 08:41:52 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v3 7/7] android: Remove Camera3RequestDescriptor::streamsProcessMutex_ Date: Wed, 4 Dec 2024 16:36:32 +0000 Message-ID: <20241204164137.3938891-8-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241204164137.3938891-1-chenghaoyang@chromium.org> References: <20241204164137.3938891-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This mutex was needed when CameraStream's worker thread posts a result back to CameraDevice. We can simplify it by calling CameraDevice's function on libcamera::Camera's owner thread. With this delegation, `Camera3RequestDescriptor::pendingStreamsToProcess_` will be firstly setup in the application's thread in processCaptureRequest(), and the rest accesses will be done in libcamera::CameraManager's thread. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 33 +++++++++++++++++---------------- src/android/camera_device.h | 6 ++++-- src/android/camera_request.h | 4 +--- src/android/camera_stream.cpp | 4 ++-- 4 files changed, 24 insertions(+), 23 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 18e974f56..0e71a9f0c 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -990,8 +990,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques FrameBuffer *frameBuffer = nullptr; UniqueFD acquireFence; - MutexLocker lock(descriptor->streamsProcessMutex_); - switch (cameraStream->type()) { case CameraStream::Type::Mapped: /* Mapped streams will be handled in the next loop. */ @@ -1067,7 +1065,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques << cameraStream->configuration().pixelFormat << "]" << " (mapped)"; - MutexLocker lock(descriptor->streamsProcessMutex_); descriptor->pendingStreamsToProcess_.insert({ cameraStream, &buffer }); /* @@ -1242,9 +1239,6 @@ void CameraDevice::requestComplete(Request *request) descriptor->resultMetadata_ = std::make_unique(0, 0); } - /* Handle post-processing. */ - MutexLocker locker(descriptor->streamsProcessMutex_); - /* * Queue all the post-processing streams request at once. The completion * slot streamProcessingComplete() can only execute when we are out @@ -1272,10 +1266,8 @@ void CameraDevice::requestComplete(Request *request) } } - if (descriptor->pendingStreamsToProcess_.empty()) { - locker.unlock(); + if (descriptor->pendingStreamsToProcess_.empty()) completeDescriptor(descriptor); - } } /** @@ -1382,6 +1374,19 @@ void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, } } +void CameraDevice::streamProcessingCompleteDelegate(StreamBuffer *streamBuffer, + StreamBuffer::Status status) +{ + /* + * Delegate the callback to the camera manager thread to simplify race condition. + */ + auto *method = new BoundMethodMember{ + this, camera_.get(), &CameraDevice::streamProcessingComplete, ConnectionTypeQueued + }; + + method->activate(streamBuffer, status); +} + /** * \brief Handle post-processing completion of a stream in a capture request * \param[in] streamBuffer The StreamBuffer for which processing is complete @@ -1402,13 +1407,9 @@ void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, Camera3RequestDescriptor *request = streamBuffer->request; - { - MutexLocker locker(request->streamsProcessMutex_); - - request->pendingStreamsToProcess_.erase(streamBuffer->stream); - if (!request->pendingStreamsToProcess_.empty()) - return; - } + request->pendingStreamsToProcess_.erase(streamBuffer->stream); + if (!request->pendingStreamsToProcess_.empty()) + return; completeDescriptor(streamBuffer->request); } diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 699aa8f17..69d163d76 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -65,8 +65,8 @@ public: int configureStreams(camera3_stream_configuration_t *stream_list); int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); - void streamProcessingComplete(StreamBuffer *bufferStream, - StreamBuffer::Status status); + void streamProcessingCompleteDelegate(StreamBuffer *bufferStream, + StreamBuffer::Status status); protected: std::string logPrefix() const override; @@ -96,6 +96,8 @@ private: int processControls(Camera3RequestDescriptor *descriptor); void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); + void streamProcessingComplete(StreamBuffer *bufferStream, + StreamBuffer::Status status); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); void sendCaptureResult(Camera3RequestDescriptor *request) const; void setBufferStatus(StreamBuffer &buffer, diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 6b2a00795..bd75d4595 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -66,9 +66,7 @@ public: }; /* Keeps track of streams requiring post-processing. */ - std::map pendingStreamsToProcess_ - LIBCAMERA_TSA_GUARDED_BY(streamsProcessMutex_); - libcamera::Mutex streamsProcessMutex_; + std::map pendingStreamsToProcess_; Camera3RequestDescriptor(libcamera::Camera *camera, const camera3_capture_request_t *camera3Request); diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 53f292d4b..7837fd7aa 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -121,8 +121,8 @@ int CameraStream::configure() else bufferStatus = StreamBuffer::Status::Error; - cameraDevice_->streamProcessingComplete(streamBuffer, - bufferStatus); + cameraDevice_->streamProcessingCompleteDelegate(streamBuffer, + bufferStatus); }); worker_->start();