From patchwork Tue Dec 10 14:23:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22269 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F50EC3257 for ; Tue, 10 Dec 2024 14:26:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C331B67E91; Tue, 10 Dec 2024 15:26:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="L102QSmN"; dkim-atps=neutral Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 63E9767E8A for ; Tue, 10 Dec 2024 15:26:05 +0100 (CET) Received: by mail-pg1-x52f.google.com with SMTP id 41be03b00d2f7-7fcfb7db9bfso4344352a12.1 for ; Tue, 10 Dec 2024 06:26:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840763; x=1734445563; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VzdjXSoSlpxLRIYtNCwI2M+hp0CGvhy+AVndgobM5W0=; b=L102QSmNFJIbRvba8ZQK3fuLhFVLrFA3h9PcbEYdOETUmaxlidqN6/wpFFVzjZ7orT NDkFmnlVdWnV/UwzpnNc9MyzrxzNrAsxs0mgNlVtjMuseEDKot2gSBaiRsdg/0xW5cuv ZqJAsNUKLFqtC8ryJHCLR+MkTiEpQh8hHmTz8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840763; x=1734445563; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VzdjXSoSlpxLRIYtNCwI2M+hp0CGvhy+AVndgobM5W0=; b=Y9XaXQN3U6rNM5NoTpco5087RsekzKx1FENKfpCATj8YDIIzLAN0Ddc31gU25RhGUY SvlQOA8woVzFI1P5sxPAmE9y/PT3b01xbpkqZPl3WTcJiiNPbxygPWocm2zOhMtV+C9G G8S0BWEK3FOTAU8q/JRQcALKmdDzeq4dyVpj9ggATQG85uaEWoiFuXie7SCQCphZN5Af +54WNQgHiGHs7ijSHXsxpJJ4lEXrKg1Uk6Ot0XR41W2M5Kx+Hqqzui4AHmafPTRwAc3q IbhsQmTmsTZzimvyU5A8xcGdJHeMQnfs0J8pRqZ6OsUAdM/pI9YxMrFBIJT6jU7mpnED IfTw== X-Gm-Message-State: AOJu0YynqC7VHYMqbO/Z1J/sbA8VJmrKtLA7MVuc0mpo+SaJBChXrnL/ 3wVeVZAUb7/mYCWQOHIoFpqssUmYlF4PmDziQNem6rCd2vxUeAXUkak/tIGxiXaTFeKR31zCnzQ = X-Gm-Gg: ASbGncsY9wPjMZeHn9VuYaXwwgohdqZqN+r5VvHkacr2vhr/RBajmrBim8LX6cXSN3W rP13kyXZ2gxqAFL62EElSmAZKlcQOnxa1lpbY+MmUAJrqQoIDdNOUG6oG9f3BlFhop8HvI7dYNy CnFSE2W2MmH9/3nvkRkx2CHuPEpFmi2+gqLmybk/CluoJD6UBABAgZzc64q3rRmgmOhaIXCkcwD RNqoJCzFgT/dYV1ctdUSFXiaQmZUK5AIM5bSLaIPHJnXgCRrwZZHHzbbKm8Zt9EalE8xyDn7hMJ 2aK7gq3HqkK7/M3JNKHDy9Pd+BRqoRKL6P+PYU3lX0ajxbdk+CbPxDqU7Vye X-Google-Smtp-Source: AGHT+IGEiQVbUxLIh6wV1QwoLeo9T+deRjni8Hu9hIzaxyiVlA+aMWHea/RTI5ED+HdZw/WOFFkVgA== X-Received: by 2002:a17:90b:510f:b0:2ee:b8ac:73b4 with SMTP id 98e67ed59e1d1-2efcf26e338mr6642498a91.36.1733840763448; Tue, 10 Dec 2024 06:26:03 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:03 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v4 1/7] android: Move StreamBuffer out of Camera3RequestDescriptor Date: Tue, 10 Dec 2024 14:23:54 +0000 Message-ID: <20241210142557.2886315-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move StreamBuffer out of Camera3RequestDescriptor before applying partial result to Android adaptor. This change makes it easier to add Camera3ResultDescriptor, which references both StreamBuffer and Camera3RequestDescriptor Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 22 +++++----- src/android/camera_device.h | 8 ++-- src/android/camera_request.cpp | 29 +++++++------ src/android/camera_request.h | 53 ++++++++++++++---------- src/android/camera_stream.cpp | 16 +++---- src/android/camera_stream.h | 6 +-- src/android/jpeg/encoder.h | 2 +- src/android/jpeg/encoder_jea.cpp | 2 +- src/android/jpeg/encoder_jea.h | 2 +- src/android/jpeg/encoder_libjpeg.cpp | 2 +- src/android/jpeg/encoder_libjpeg.h | 2 +- src/android/jpeg/post_processor_jpeg.cpp | 2 +- src/android/jpeg/post_processor_jpeg.h | 2 +- src/android/post_processor.h | 4 +- src/android/yuv/post_processor_yuv.cpp | 2 +- src/android/yuv/post_processor_yuv.h | 2 +- 16 files changed, 82 insertions(+), 74 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a038131ae..4e3bdc9cc 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -865,7 +865,7 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); for (auto &buffer : descriptor->buffers_) - buffer.status = Camera3RequestDescriptor::Status::Error; + buffer.status = StreamBuffer::Status::Error; descriptor->status_ = Camera3RequestDescriptor::Status::Error; } @@ -1163,7 +1163,7 @@ void CameraDevice::requestComplete(Request *request) if (fence) buffer.fence = fence->release(); } - buffer.status = Camera3RequestDescriptor::Status::Success; + buffer.status = StreamBuffer::Status::Success; } /* @@ -1226,12 +1226,12 @@ void CameraDevice::requestComplete(Request *request) auto iter = descriptor->pendingStreamsToProcess_.begin(); while (iter != descriptor->pendingStreamsToProcess_.end()) { CameraStream *stream = iter->first; - Camera3RequestDescriptor::StreamBuffer *buffer = iter->second; + StreamBuffer *buffer = iter->second; FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; - setBufferStatus(*buffer, Camera3RequestDescriptor::Status::Error); + setBufferStatus(*buffer, StreamBuffer::Status::Error); iter = descriptor->pendingStreamsToProcess_.erase(iter); continue; } @@ -1241,7 +1241,7 @@ void CameraDevice::requestComplete(Request *request) ++iter; int ret = stream->process(buffer); if (ret) { - setBufferStatus(*buffer, Camera3RequestDescriptor::Status::Error); + setBufferStatus(*buffer, StreamBuffer::Status::Error); descriptor->pendingStreamsToProcess_.erase(stream); /* @@ -1311,7 +1311,7 @@ void CameraDevice::sendCaptureResults() for (auto &buffer : descriptor->buffers_) { camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; - if (buffer.status == Camera3RequestDescriptor::Status::Success) + if (buffer.status == StreamBuffer::Status::Success) status = CAMERA3_BUFFER_STATUS_OK; /* @@ -1335,11 +1335,11 @@ void CameraDevice::sendCaptureResults() } } -void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &streamBuffer, - Camera3RequestDescriptor::Status status) +void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, + StreamBuffer::Status status) { streamBuffer.status = status; - if (status != Camera3RequestDescriptor::Status::Success) { + if (status != StreamBuffer::Status::Success) { notifyError(streamBuffer.request->frameNumber_, streamBuffer.stream->camera3Stream(), CAMERA3_MSG_ERROR_BUFFER); @@ -1363,8 +1363,8 @@ void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &strea * be generated from post-processing have been completed. Mark the descriptor as * complete using completeDescriptor() in that case. */ -void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *streamBuffer, - Camera3RequestDescriptor::Status status) +void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, + StreamBuffer::Status status) { setBufferStatus(*streamBuffer, status); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 194ca3030..c92ee1aa4 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -65,8 +65,8 @@ public: int configureStreams(camera3_stream_configuration_t *stream_list); int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); - void streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *bufferStream, - Camera3RequestDescriptor::Status status); + void streamProcessingComplete(StreamBuffer *bufferStream, + StreamBuffer::Status status); protected: std::string logPrefix() const override; @@ -97,8 +97,8 @@ private: void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); - void setBufferStatus(Camera3RequestDescriptor::StreamBuffer &buffer, - Camera3RequestDescriptor::Status status); + void setBufferStatus(StreamBuffer &buffer, + StreamBuffer::Status status); std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 0d45960d9..52a3ac1f7 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -141,7 +141,7 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; /** - * \struct Camera3RequestDescriptor::StreamBuffer + * \class StreamBuffer * \brief Group information for per-stream buffer of Camera3RequestDescriptor * * A capture request placed to the libcamera HAL can contain multiple streams. @@ -150,35 +150,35 @@ Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; * generation. The generation of the stream will depend on its type (refer to * the CameraStream::Type documentation). * - * \var Camera3RequestDescriptor::StreamBuffer::stream + * \var StreamBuffer::stream * \brief Pointer to the corresponding CameraStream * - * \var Camera3RequestDescriptor::StreamBuffer::camera3Buffer + * \var StreamBuffer::camera3Buffer * \brief Native handle to the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::frameBuffer + * \var StreamBuffer::frameBuffer * \brief Encapsulate the dmabuf handle inside a libcamera::FrameBuffer for * direct streams * - * \var Camera3RequestDescriptor::StreamBuffer::fence + * \var StreamBuffer::fence * \brief Acquire fence of the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::status + * \var StreamBuffer::status * \brief Track the status of the buffer * - * \var Camera3RequestDescriptor::StreamBuffer::internalBuffer + * \var StreamBuffer::internalBuffer * \brief Pointer to a buffer internally handled by CameraStream (if any) * - * \var Camera3RequestDescriptor::StreamBuffer::srcBuffer + * \var StreamBuffer::srcBuffer * \brief Pointer to the source frame buffer used for post-processing * - * \var Camera3RequestDescriptor::StreamBuffer::dstBuffer + * \var StreamBuffer::dstBuffer * \brief Pointer to the destination frame buffer used for post-processing * - * \var Camera3RequestDescriptor::StreamBuffer::request + * \var StreamBuffer::request * \brief Back pointer to the Camera3RequestDescriptor to which the StreamBuffer belongs */ -Camera3RequestDescriptor::StreamBuffer::StreamBuffer( +StreamBuffer::StreamBuffer( CameraStream *cameraStream, const camera3_stream_buffer_t &buffer, Camera3RequestDescriptor *requestDescriptor) : stream(cameraStream), camera3Buffer(buffer.buffer), @@ -186,9 +186,8 @@ Camera3RequestDescriptor::StreamBuffer::StreamBuffer( { } -Camera3RequestDescriptor::StreamBuffer::~StreamBuffer() = default; +StreamBuffer::~StreamBuffer() = default; -Camera3RequestDescriptor::StreamBuffer::StreamBuffer(StreamBuffer &&) = default; +StreamBuffer::StreamBuffer(StreamBuffer &&) = default; -Camera3RequestDescriptor::StreamBuffer & -Camera3RequestDescriptor::StreamBuffer::operator=(Camera3RequestDescriptor::StreamBuffer &&) = default; +StreamBuffer &StreamBuffer::operator=(StreamBuffer &&) = default; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 5b479180f..335f1985d 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -26,7 +26,9 @@ class CameraBuffer; class CameraStream; -class Camera3RequestDescriptor +class Camera3RequestDescriptor; + +class StreamBuffer { public: enum class Status { @@ -34,27 +36,34 @@ public: Error, }; - struct StreamBuffer { - StreamBuffer(CameraStream *stream, - const camera3_stream_buffer_t &buffer, - Camera3RequestDescriptor *request); - ~StreamBuffer(); - - StreamBuffer(StreamBuffer &&); - StreamBuffer &operator=(StreamBuffer &&); - - CameraStream *stream; - buffer_handle_t *camera3Buffer; - std::unique_ptr frameBuffer; - libcamera::UniqueFD fence; - Status status = Status::Success; - libcamera::FrameBuffer *internalBuffer = nullptr; - const libcamera::FrameBuffer *srcBuffer = nullptr; - std::unique_ptr dstBuffer; - Camera3RequestDescriptor *request; - - private: - LIBCAMERA_DISABLE_COPY(StreamBuffer) + StreamBuffer(CameraStream *stream, + const camera3_stream_buffer_t &buffer, + Camera3RequestDescriptor *request); + ~StreamBuffer(); + + StreamBuffer(StreamBuffer &&); + StreamBuffer &operator=(StreamBuffer &&); + + CameraStream *stream; + buffer_handle_t *camera3Buffer; + std::unique_ptr frameBuffer; + libcamera::UniqueFD fence; + Status status = Status::Success; + libcamera::FrameBuffer *internalBuffer = nullptr; + const libcamera::FrameBuffer *srcBuffer = nullptr; + std::unique_ptr dstBuffer; + Camera3RequestDescriptor *request; + +private: + LIBCAMERA_DISABLE_COPY(StreamBuffer) +}; + +class Camera3RequestDescriptor +{ +public: + enum class Status { + Success, + Error, }; /* Keeps track of streams requiring post-processing. */ diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 1d68540d7..53f292d4b 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -112,14 +112,14 @@ int CameraStream::configure() worker_ = std::make_unique(postProcessor_.get()); postProcessor_->processComplete.connect( - this, [&](Camera3RequestDescriptor::StreamBuffer *streamBuffer, + this, [&](StreamBuffer *streamBuffer, PostProcessor::Status status) { - Camera3RequestDescriptor::Status bufferStatus; + StreamBuffer::Status bufferStatus; if (status == PostProcessor::Status::Success) - bufferStatus = Camera3RequestDescriptor::Status::Success; + bufferStatus = StreamBuffer::Status::Success; else - bufferStatus = Camera3RequestDescriptor::Status::Error; + bufferStatus = StreamBuffer::Status::Error; cameraDevice_->streamProcessingComplete(streamBuffer, bufferStatus); @@ -165,7 +165,7 @@ int CameraStream::waitFence(int fence) return -errno; } -int CameraStream::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +int CameraStream::process(StreamBuffer *streamBuffer) { ASSERT(type_ != Type::Direct); @@ -283,7 +283,7 @@ void CameraStream::PostProcessorWorker::start() Thread::start(); } -void CameraStream::PostProcessorWorker::queueRequest(Camera3RequestDescriptor::StreamBuffer *dest) +void CameraStream::PostProcessorWorker::queueRequest(StreamBuffer *dest) { { MutexLocker lock(mutex_); @@ -306,7 +306,7 @@ void CameraStream::PostProcessorWorker::run() if (state_ != State::Running) break; - Camera3RequestDescriptor::StreamBuffer *streamBuffer = requests_.front(); + StreamBuffer *streamBuffer = requests_.front(); requests_.pop(); locker.unlock(); @@ -316,7 +316,7 @@ void CameraStream::PostProcessorWorker::run() } if (state_ == State::Flushing) { - std::queue requests = + std::queue requests = std::move(requests_); locker.unlock(); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 395552dac..30f64f690 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -127,7 +127,7 @@ public: CameraStream *sourceStream() const { return sourceStream_; } int configure(); - int process(Camera3RequestDescriptor::StreamBuffer *streamBuffer); + int process(StreamBuffer *streamBuffer); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); void flush(); @@ -146,7 +146,7 @@ private: ~PostProcessorWorker(); void start(); - void queueRequest(Camera3RequestDescriptor::StreamBuffer *request); + void queueRequest(StreamBuffer *request); void flush(); protected: @@ -158,7 +158,7 @@ private: libcamera::Mutex mutex_; libcamera::ConditionVariable cv_; - std::queue requests_ + std::queue requests_ LIBCAMERA_TSA_GUARDED_BY(mutex_); State state_ LIBCAMERA_TSA_GUARDED_BY(mutex_) = State::Stopped; diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h index ed033c191..25425d92c 100644 --- a/src/android/jpeg/encoder.h +++ b/src/android/jpeg/encoder.h @@ -20,7 +20,7 @@ public: virtual ~Encoder() = default; virtual int configure(const libcamera::StreamConfiguration &cfg) = 0; - virtual int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + virtual int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) = 0; }; diff --git a/src/android/jpeg/encoder_jea.cpp b/src/android/jpeg/encoder_jea.cpp index 25dc43173..b3ad04638 100644 --- a/src/android/jpeg/encoder_jea.cpp +++ b/src/android/jpeg/encoder_jea.cpp @@ -33,7 +33,7 @@ int EncoderJea::configure(const libcamera::StreamConfiguration &cfg) return 0; } -int EncoderJea::encode(Camera3RequestDescriptor::StreamBuffer *buffer, +int EncoderJea::encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) { diff --git a/src/android/jpeg/encoder_jea.h b/src/android/jpeg/encoder_jea.h index 91115d2e9..79265ebc6 100644 --- a/src/android/jpeg/encoder_jea.h +++ b/src/android/jpeg/encoder_jea.h @@ -20,7 +20,7 @@ public: ~EncoderJea(); int configure(const libcamera::StreamConfiguration &cfg) override; - int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) override; diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp index cb242b5ec..cb123576a 100644 --- a/src/android/jpeg/encoder_libjpeg.cpp +++ b/src/android/jpeg/encoder_libjpeg.cpp @@ -180,7 +180,7 @@ void EncoderLibJpeg::compressNV(const std::vector> &planes) } } -int EncoderLibJpeg::encode(Camera3RequestDescriptor::StreamBuffer *buffer, +int EncoderLibJpeg::encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) { diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h index 4ac85c22e..de04b17d1 100644 --- a/src/android/jpeg/encoder_libjpeg.h +++ b/src/android/jpeg/encoder_libjpeg.h @@ -22,7 +22,7 @@ public: ~EncoderLibJpeg(); int configure(const libcamera::StreamConfiguration &cfg) override; - int encode(Camera3RequestDescriptor::StreamBuffer *buffer, + int encode(StreamBuffer *buffer, libcamera::Span exifData, unsigned int quality) override; int encode(const std::vector> &planes, diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index 89b8a401e..f5a90785d 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -106,7 +106,7 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, } } -void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +void PostProcessorJpeg::process(StreamBuffer *streamBuffer) { ASSERT(encoder_); diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index 6fe214577..5421f233e 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -22,7 +22,7 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override; + void process(StreamBuffer *streamBuffer) override; private: void generateThumbnail(const libcamera::FrameBuffer &source, diff --git a/src/android/post_processor.h b/src/android/post_processor.h index b504a3796..f2e00d2b1 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -27,7 +27,7 @@ public: virtual int configure(const libcamera::StreamConfiguration &inCfg, const libcamera::StreamConfiguration &outCfg) = 0; - virtual void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) = 0; + virtual void process(StreamBuffer *streamBuffer) = 0; - libcamera::Signal processComplete; + libcamera::Signal processComplete; }; diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index c998807b0..b05623b23 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -49,7 +49,7 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, return 0; } -void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) +void PostProcessorYuv::process(StreamBuffer *streamBuffer) { const FrameBuffer &source = *streamBuffer->srcBuffer; CameraBuffer *destination = streamBuffer->dstBuffer.get(); diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index ed7bb1fbe..459c6685e 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -18,7 +18,7 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override; + void process(StreamBuffer *streamBuffer) override; private: bool isValidBuffers(const libcamera::FrameBuffer &source, From patchwork Tue Dec 10 14:23:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22270 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9A93EC3257 for ; Tue, 10 Dec 2024 14:26:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AF6F67E90; Tue, 10 Dec 2024 15:26:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="h5eX9RwO"; dkim-atps=neutral Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1E9967E85 for ; Tue, 10 Dec 2024 15:26:06 +0100 (CET) Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2ee76929a98so4409264a91.1 for ; Tue, 10 Dec 2024 06:26:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840765; x=1734445565; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7RSUBsgMlBRQcCiQn/OQffu9bSxAh/8WGY8/nToqTB8=; b=h5eX9RwOsDCzRlFcxaZwTRyvkbvklv7RTOSZ+tJc9BBhB6ZI9YJfymmNw9O2JB4y30 NL/g2RSCA+YNdwOZzoK7iLWORQuZr08OuWDtCQXmdPtmEt+O3+w5Z5PIzSqFiRtVoWah eRjj+0cEeVvZqQLGmtzhbK0C57ZyKmqZpijuA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840765; x=1734445565; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7RSUBsgMlBRQcCiQn/OQffu9bSxAh/8WGY8/nToqTB8=; b=QmEr6y4GdBXRG0OubaXi96l0B+4oa5O1eKXxnHya9yZ+DDocV1Umym6bzwqXvb+Pg2 4FEx1TXUB2AaTrnq+0ql+/+wEjxLK6Ji2ZDUEVhlUZZNRtysCvDZQQ0vNk0XK5zhaC9f u+a34C7MBhSgrDAtnfvosrH57i7rgD00JHNVWboxebjWsI9IaEG9E0udqwUitXoHXAeo 5LIfAMxetHU1bFnjZrsWDQaKpROQ/s29OZ0HyHNa8K6m8CWR4KudSdGbVkD2kfSdbYhC 82CppxYn5zy/fdgiKf8pJxpYkrvWmsAYwSPyqxUm5BTmDmInQc1WYHgK4QfwptYdOET7 xXug== X-Gm-Message-State: AOJu0YwWjjADhhWYxIQ2BDcBe3W61cuWIhuJNlscmBb5Ql4eSqnn579V ZIA98CLdS2LcmaitAVqSOedlN8cVS/I9mbjjKjeL16UU0zAJufwIFxh24su5bM0G6u8kfxwZoRc = X-Gm-Gg: ASbGncs/bQsvTBY50Zdx3HIMe1Pun74ozTj1CoumDV/xYfwVd/KTXBoWMzS2OmcNQex 13rf6jAaPsTR1OyPxunaYODyA7CfYNDpxb2B5ci03vSRZXdBoCBItuHhIExH8s7HxTDPrd+BhtX qzK6Y51DkeGhI/D1vsagPT5gtZGqfl1xgt129QosIRa+ihsuvcTubJgLDQKK4xHIKaanKpvsEP3 wNWkANh+wplqqdGmn/jm6OPfwaQaRzOsxldCnhpG049oU1RR3Y6ltP4yZC57raAUGwIXNyDkZTL eopgfE0GaJMdIIRvNg+GW4qjf7RSGVODJgcjdyxybtAkxZ7s6eSagEdxWKYS X-Google-Smtp-Source: AGHT+IGaw2zjiZPuu8FAyKww2a8rQvWXahex/o8HhofuiRnIMe4kuxYH2faKSsmsZYOEX55D3wrO3A== X-Received: by 2002:a17:90b:17c8:b0:2ee:ee77:226d with SMTP id 98e67ed59e1d1-2efd597dfb8mr5339886a91.4.1733840764875; Tue, 10 Dec 2024 06:26:04 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:04 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v4 2/7] android: Set StreamBuffer::srcBuffer in CameraDevice::processCaptureRequest Date: Tue, 10 Dec 2024 14:23:55 +0000 Message-ID: <20241210142557.2886315-3-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamBuffer::srcBuffer was set right before being processed, while it was already determined when being constructed. This patch sets the value in CameraDevice::processCaptureRequest. `requestedStreams` is refactored from std::set to std::map to find the correct srcBuffer accordingly. The change also helps cleaning up the post process pipeline's dependency on `Camera3RequestDescriptor::pendingStreamsToProcess_`, which is going to be cleaned up in the following patches. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 4e3bdc9cc..f6dadaf22 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -9,7 +9,6 @@ #include #include -#include #include #include #include @@ -967,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * to a libcamera stream. Streams of type Mapped will be handled later. * * Collect the CameraStream associated to each requested capture stream. - * Since requestedStreams is an std:set<>, no duplications can happen. + * Since requestedStreams is an std:map<>, no duplications can happen. */ - std::set requestedStreams; + std::map requestedBuffers; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -1021,6 +1020,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques */ frameBuffer = cameraStream->getBuffer(); buffer.internalBuffer = frameBuffer; + buffer.srcBuffer = frameBuffer; + LOG(HAL, Debug) << ss.str() << " (internal)"; descriptor->pendingStreamsToProcess_.insert( @@ -1037,7 +1038,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); - requestedStreams.insert(cameraStream); + requestedBuffers[cameraStream] = frameBuffer; } /* @@ -1067,9 +1068,19 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * added to the request. */ CameraStream *sourceStream = cameraStream->sourceStream(); - ASSERT(sourceStream); - if (requestedStreams.find(sourceStream) != requestedStreams.end()) + ASSERT(sourceStream && sourceStream->type() == CameraStream::Type::Direct); + + /* + * If the buffer for the source stream has been requested as + * Direct, use its framebuffer as the source buffer for + * post-processing. No need to recycle the buffer since it's + * owned by Android. + */ + auto iterDirectBuffer = requestedBuffers.find(sourceStream); + if (iterDirectBuffer != requestedBuffers.end()) { + buffer.srcBuffer = iterDirectBuffer->second; continue; + } /* * If that's not the case, we need to add a buffer to the request @@ -1077,11 +1088,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques */ FrameBuffer *frameBuffer = cameraStream->getBuffer(); buffer.internalBuffer = frameBuffer; + buffer.srcBuffer = frameBuffer; descriptor->request_->addBuffer(sourceStream->stream(), frameBuffer, nullptr); - requestedStreams.insert(sourceStream); + requestedBuffers[sourceStream] = frameBuffer; } /* @@ -1236,8 +1248,6 @@ void CameraDevice::requestComplete(Request *request) continue; } - buffer->srcBuffer = src; - ++iter; int ret = stream->process(buffer); if (ret) { From patchwork Tue Dec 10 14:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22271 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C2996C3257 for ; Tue, 10 Dec 2024 14:26:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D37B67E92; Tue, 10 Dec 2024 15:26:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="fjuObykP"; dkim-atps=neutral Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40ACB67E8D for ; Tue, 10 Dec 2024 15:26:08 +0100 (CET) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2efded08c79so458098a91.0 for ; Tue, 10 Dec 2024 06:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840766; x=1734445566; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MbONCrxxlu5qaHg94AwjcNNxPrpZW9pFEh4Xyevk5rs=; b=fjuObykPNGo5tl8PKXzoumtT2XA53zsP7WEa8okWSOxbpkS/bmhew/3KR5wMBIvlAl mAh/610toZnb21r2xNcviqs+ghTJJzBVEob4yqq2mFzhjbriubb8YaNEgUZFBf0CGnzn DPni0673xV/RZ0+l4AjHc1pvxfcsFPZ+dX698= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840766; x=1734445566; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MbONCrxxlu5qaHg94AwjcNNxPrpZW9pFEh4Xyevk5rs=; b=RBG87gSyKWw5ScOypBh/aWTYG3xv2g8o8pxjkGQfR7fOp/EFxx9hRFni7vrxr7pCn4 l09Qj8LqjaGoG+/BKm7N3sb6FDTq6pRqR6qFW6NxQqO7iMc7T3glEl876tfVTBDSN3gT jN5Rnx6ivkmmQ5uQ+vPAFes9Re1Oj/XFUOaikL73pM+mIC3jK01vCQrS1n6sVxWXxo6h 4ukEKg8KUB5ddt39HB4gxN5jxXAYJ+2/6mxHOfpGKrqM/9re/h6zzwS8m4s9Hd9IyxWU iYtQNxBxWFkhEY42wKsSmrlLqQTtvPKV65KcpQQYsh/Q/lwBNhqy1NV1mtXVT9q7AiiD remg== X-Gm-Message-State: AOJu0YwZYHGbSjQB0P8erpsHXDcGTSyAZ5Fv7sdTLTc2H6Czqndq4mZx w7eyNhcYtwT1buc2ajaSY36dvo9MOigcR6BXx5/2oV1tqpTWDePz1fmdrhurbP1dLlxgH3sNbiM = X-Gm-Gg: ASbGnctt9aP9p8jRTHYBNS5RWg+sVO8K6v3sLXtPISwHRpjSivgq55l59hiHixd0PDa c56cN5gFvQu1kECbLQToqT3JWRJXX7grzFr7nH0jRJdQ0qplacw0uaMZ9BvRidJIQ4+/EO5dISV LCCMGM7O5x9A4R+0XwwEL1ZwqBMCt5rmJ6X8Px0N0O+cZpV6kEK1/wY+2dLVIkpKS91wZ28F+ka 2yroRsFo84Ghy5RKBktGh8+AgHizbcaHcWbHC02WVszXvh6TCf8VEJvgpRr6UexZ/8hzFmjsbU1 ulJPQQ98du0UVPprHZS+nuYOSn8kZxCymB6sVOVKWdnFzrbsU1WWvsHrqFZb X-Google-Smtp-Source: AGHT+IEUl5BISdsGpljYIFVeCZOVUu1qDBp8MgWvWI2WyWvnTSuR+FJcpBy2njOQkE2RA71z5lXuiw== X-Received: by 2002:a17:90b:5102:b0:2ef:e0bb:1ef2 with SMTP id 98e67ed59e1d1-2efe0bb1ff5mr2676540a91.19.1733840766405; Tue, 10 Dec 2024 06:26:06 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:06 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v4 3/7] android: Correctly support multiple Mapped streams Date: Tue, 10 Dec 2024 14:23:56 +0000 Message-ID: <20241210142557.2886315-4-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Android camera HAL supports creating streams for the Android framework by post-processing streams produced by libcamera. However at the moment a single Mapped stream can be associated with a Direct stream because, after the first post-processing, the data from the internal stream are returned preventing further post-processing passes. Fix this by storing the list of Internal stream buffers used as the post-processing source in an Camera3RequestDescriptor::internalBuffers_ map to replace the single internalBuffer_ pointer and return the internal buffers when the capture request is deleted. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 65 +++++++++++++++++++--------------- src/android/camera_request.cpp | 11 +++--- src/android/camera_request.h | 3 +- 3 files changed, 46 insertions(+), 33 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f6dadaf22..6bc19a74a 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -966,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * to a libcamera stream. Streams of type Mapped will be handled later. * * Collect the CameraStream associated to each requested capture stream. - * Since requestedStreams is an std:map<>, no duplications can happen. + * Since directBuffers is an std:map<>, no duplications can happen. */ - std::map requestedBuffers; + std::map directBuffers; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -1007,6 +1007,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques cameraStream->configuration().size); frameBuffer = buffer.frameBuffer.get(); acquireFence = std::move(buffer.fence); + + directBuffers[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -1015,13 +1017,17 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * Get the frame buffer from the CameraStream internal * buffer pool. * - * The buffer has to be returned to the CameraStream - * once it has been processed. + * The buffer will be returned to the CameraStream when + * the request is destroyed. */ frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; buffer.srcBuffer = frameBuffer; + /* + * Track the allocated internal buffers, which will be + * recycled when the descriptor is destroyed. + */ + descriptor->internalBuffers_[cameraStream] = frameBuffer; LOG(HAL, Debug) << ss.str() << " (internal)"; descriptor->pendingStreamsToProcess_.insert( @@ -1037,8 +1043,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques auto fence = std::make_unique(std::move(acquireFence)); descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); - - requestedBuffers[cameraStream] = frameBuffer; } /* @@ -1076,24 +1080,43 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * post-processing. No need to recycle the buffer since it's * owned by Android. */ - auto iterDirectBuffer = requestedBuffers.find(sourceStream); - if (iterDirectBuffer != requestedBuffers.end()) { + auto iterDirectBuffer = directBuffers.find(sourceStream); + if (iterDirectBuffer != directBuffers.end()) { buffer.srcBuffer = iterDirectBuffer->second; continue; } /* - * If that's not the case, we need to add a buffer to the request - * for this stream. + * If that's not the case, we use an internal buffer allocated + * from the source stream. + */ + + /* + * If an internal buffer has been requested for the source + * stream before, we should reuse it. */ - FrameBuffer *frameBuffer = cameraStream->getBuffer(); - buffer.internalBuffer = frameBuffer; + auto iterInternalBuffer = descriptor->internalBuffers_.find(sourceStream); + if (iterInternalBuffer != descriptor->internalBuffers_.end()) { + buffer.srcBuffer = iterInternalBuffer->second; + continue; + } + + /* + * Otherwise, we need to create an internal buffer to the + * request for the source stream. Get the frame buffer from the + * source stream's internal buffer pool. + * + * The buffer will be returned to the CameraStream when the + * request is destructed. + */ + FrameBuffer *frameBuffer = sourceStream->getBuffer(); buffer.srcBuffer = frameBuffer; descriptor->request_->addBuffer(sourceStream->stream(), frameBuffer, nullptr); - requestedBuffers[sourceStream] = frameBuffer; + /* Track the allocated internal buffer. */ + descriptor->internalBuffers_[sourceStream] = frameBuffer; } /* @@ -1253,13 +1276,6 @@ void CameraDevice::requestComplete(Request *request) if (ret) { setBufferStatus(*buffer, StreamBuffer::Status::Error); descriptor->pendingStreamsToProcess_.erase(stream); - - /* - * If the framebuffer is internal to CameraStream return - * it back now that we're done processing it. - */ - if (buffer->internalBuffer) - stream->putBuffer(buffer->internalBuffer); } } @@ -1378,13 +1394,6 @@ void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, { setBufferStatus(*streamBuffer, status); - /* - * If the framebuffer is internal to CameraStream return it back now - * that we're done processing it. - */ - if (streamBuffer->internalBuffer) - streamBuffer->stream->putBuffer(streamBuffer->internalBuffer); - Camera3RequestDescriptor *request = streamBuffer->request; { diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 52a3ac1f7..92e74ab6a 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -10,6 +10,7 @@ #include #include "camera_buffer.h" +#include "camera_stream.h" using namespace libcamera; @@ -138,7 +139,12 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( request_ = camera->createRequest(reinterpret_cast(this)); } -Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; +Camera3RequestDescriptor::~Camera3RequestDescriptor() +{ + /* Recycle the allocated internal buffer back to its source stream. */ + for (auto &[sourceStream, frameBuffer] : internalBuffers_) + sourceStream->putBuffer(frameBuffer); +} /** * \class StreamBuffer @@ -166,9 +172,6 @@ Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; * \var StreamBuffer::status * \brief Track the status of the buffer * - * \var StreamBuffer::internalBuffer - * \brief Pointer to a buffer internally handled by CameraStream (if any) - * * \var StreamBuffer::srcBuffer * \brief Pointer to the source frame buffer used for post-processing * diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 335f1985d..6b2a00795 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -49,7 +49,6 @@ public: std::unique_ptr frameBuffer; libcamera::UniqueFD fence; Status status = Status::Success; - libcamera::FrameBuffer *internalBuffer = nullptr; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; Camera3RequestDescriptor *request; @@ -85,6 +84,8 @@ public: std::unique_ptr request_; std::unique_ptr resultMetadata_; + std::map internalBuffers_; + bool complete_ = false; Status status_ = Status::Success; From patchwork Tue Dec 10 14:23:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22272 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BBF03C32DC for ; Tue, 10 Dec 2024 14:26:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C59B67E94; Tue, 10 Dec 2024 15:26:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="dhXFYs1L"; dkim-atps=neutral Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B47A967E92 for ; Tue, 10 Dec 2024 15:26:09 +0100 (CET) Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2ef6af22ea8so3851186a91.0 for ; Tue, 10 Dec 2024 06:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840768; x=1734445568; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q96BeC7n8E+Cipc4Avk4B1MuW6eoeZ2GDO3TPgfwR30=; b=dhXFYs1LrB0+HBSg2q4L9eooFL183G9znQoZ+gGhPuZa2A6m9rYjlPLYwIsLTZye7s dSoKB3EXHxYYS/5m/jTu9eg2lYg/xdkbOu8GTZLFhKMUkSMNZHkQZkgEXDzLK7Ih9k6N 09nX8tJIT+FM0J3Ey8Uerdh8rpab6ngmdtiBY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840768; x=1734445568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q96BeC7n8E+Cipc4Avk4B1MuW6eoeZ2GDO3TPgfwR30=; b=vtmebzs8ZomaPNO2jK+/5AVQSfOE5pBgCrH7YqKaANAXlMJBo9us8FSHV4CelQvEvi YxK/GgzUdT0sGso/UQTCuWTT3t/+xtIpqVRUuwKYuuAictqSoqTdbNgzLoGOEDslt/dB fJjkaqChCuubwVNrKiOvd2m9fWajvGNDYDJrT9+d+Tb+S/6dyAb5xHZ5I8KtRNh9cFro GijMY8MgwwSr/2C/Vh3hJljsSrzZugok/w8xxAMCeNycB6CcSZTNhttW7msCCLO4DN8k PKIIFqDDkWWnjStZTFP2T12RzzzVKNailCUq4CIxBUc9S/uOoRTj1OMp7z7Ysyq7YLlz ivbA== X-Gm-Message-State: AOJu0YyirYG3sgl7im6MAcUSg1jT3uGOvpP9y8qu5EYxK4QvBcm6Y3yz PoWx0bI4eKZEdUSXmcJTVUFpUkvYyvwDa0dIgKtH+/HTtlyXkBai8OQ733gFpvefcD2MVPVALLk = X-Gm-Gg: ASbGncsNoQ4hwisLcYxjueC0hjWVoJykFZS4oHQcfco0wIRoXqHOVWstWSJEmitwL+w /uyY+vPdUfBvKuJXG3EiFZWD7YRFeMM5VZ+9HOMgJdhK7PDgP8xOmA7UzB+wOChZOUNtn8EHzJR qoEC72LiBqaQxpngWMi28hDC9E8SKdpn1Y0eOqz1IT/1ZFNwbccifchfRTAZVwXaxtb5J2cLLP/ z+PMdN8k840yYxlM1S6XImXLOXlfTVBdjMErGZmRfrSNmvAKt3yGVq34QXE9MWcSRtVP/7Iuilh /XNKguDvMM/anBOn37wp2AAKCi6ZpUqQFW5Jys7kAj8LaRUqUwvMhSDYh+G9 X-Google-Smtp-Source: AGHT+IGXJGeS6mtX+n8Fmp/JnA6e9xzkj/6LwVm3r8PN+IxNudp/V6wxUB52SN08DDV7WPOm+L/EDQ== X-Received: by 2002:a17:90b:4b10:b0:2ee:535f:ac27 with SMTP id 98e67ed59e1d1-2efcf2c572dmr7671921a91.33.1733840767895; Tue, 10 Dec 2024 06:26:07 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:07 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v4 4/7] android: Add CameraDevice::sendCaptureResult() Date: Tue, 10 Dec 2024 14:23:57 +0000 Message-ID: <20241210142557.2886315-5-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The new function is separated from sendCaptureResults(), which allows other functions to send a result out of order, like aborting. It'll be updated in the upcoming patch, when the usage of error result is separated. This function also allows the upcoming partial results to be sent back to the application without waiting for the requests. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 57 ++++++++++++++++++----------------- src/android/camera_device.h | 1 + 2 files changed, 31 insertions(+), 27 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 6bc19a74a..f3f570544 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1323,42 +1323,45 @@ void CameraDevice::sendCaptureResults() auto descriptor = std::move(descriptors_.front()); descriptors_.pop(); - camera3_capture_result_t captureResult = {}; + sendCaptureResult(descriptor.get()); + } +} - captureResult.frame_number = descriptor->frameNumber_; +void CameraDevice::sendCaptureResult(Camera3RequestDescriptor *request) const +{ + std::vector resultBuffers; + resultBuffers.reserve(request->buffers_.size()); - if (descriptor->resultMetadata_) - captureResult.result = - descriptor->resultMetadata_->getMetadata(); + for (auto &buffer : request->buffers_) { + camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; - std::vector resultBuffers; - resultBuffers.reserve(descriptor->buffers_.size()); + if (buffer.status == StreamBuffer::Status::Success) + status = CAMERA3_BUFFER_STATUS_OK; - for (auto &buffer : descriptor->buffers_) { - camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; + /* + * Pass the buffer fence back to the camera framework as + * a release fence. This instructs the framework to wait + * on the acquire fence in case we haven't done so + * ourselves for any reason. + */ + resultBuffers.push_back({ buffer.stream->camera3Stream(), + buffer.camera3Buffer, status, + -1, buffer.fence.release() }); + } - if (buffer.status == StreamBuffer::Status::Success) - status = CAMERA3_BUFFER_STATUS_OK; + camera3_capture_result_t captureResult = {}; - /* - * Pass the buffer fence back to the camera framework as - * a release fence. This instructs the framework to wait - * on the acquire fence in case we haven't done so - * ourselves for any reason. - */ - resultBuffers.push_back({ buffer.stream->camera3Stream(), - buffer.camera3Buffer, status, - -1, buffer.fence.release() }); - } + captureResult.frame_number = request->frameNumber_; + captureResult.num_output_buffers = resultBuffers.size(); + captureResult.output_buffers = resultBuffers.data(); - captureResult.num_output_buffers = resultBuffers.size(); - captureResult.output_buffers = resultBuffers.data(); + if (request->status_ == Camera3RequestDescriptor::Status::Success) + captureResult.partial_result = 1; - if (descriptor->status_ == Camera3RequestDescriptor::Status::Success) - captureResult.partial_result = 1; + if (request->resultMetadata_) + captureResult.result = request->resultMetadata_->getMetadata(); - callbacks_->process_capture_result(callbacks_, &captureResult); - } + callbacks_->process_capture_result(callbacks_, &captureResult); } void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index c92ee1aa4..699aa8f17 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -97,6 +97,7 @@ private: void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); + void sendCaptureResult(Camera3RequestDescriptor *request) const; void setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status); std::unique_ptr getResultMetadata( From patchwork Tue Dec 10 14:23:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22273 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CBA51C3257 for ; Tue, 10 Dec 2024 14:26:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F34367E9E; Tue, 10 Dec 2024 15:26:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="eO/YnBEW"; dkim-atps=neutral Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 943F167E99 for ; Tue, 10 Dec 2024 15:26:12 +0100 (CET) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2ee397a82f6so4978189a91.2 for ; Tue, 10 Dec 2024 06:26:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840771; x=1734445571; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6LRoqne7Vs2NH7WovLgxTUdHOeemlMfXK+jCzHW5XUQ=; b=eO/YnBEW8zRxBaf1ML8sL9mnyKKRdBKuEasvDbAliMp5yUNanuSQDHgFI5xHRyQwn9 XPrm3bsGIimpaxlSCnmrp+dZX7UYRnsh7RDWgC4d27o0N6fFcfG1P9gAbICg+YW51oef LdjVpnM33CuasOAvKXtzUlbWl0VVYKbAMvw3c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840771; x=1734445571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6LRoqne7Vs2NH7WovLgxTUdHOeemlMfXK+jCzHW5XUQ=; b=X0ZW6z482dZGa/esH4w0Rd1Zy4vLNlvK/5cONLdh3Jd8mUZKW8fqqZh7fQeepEjreU wdL7mum8mBdhoRjPfjHkmGTSrIhDxEl1CUR2HQpEcwZ7KJjij5oqutobiSp06D4T2MRw gpwnJHHN3X23kYBWiyVS1m7+XOSJcOY8NMLFFvLOSx53fHGU7vLmickUS6VcHh/KkyDt tzmhxKT50ZS8r9rBOXJvjwybOsxYb4hbyagCoDYUjKym3M1RAXyDoBiOPB89tMiSWKZt Rir8HMJ04MBFUIf/MTv5tlNip+jp2hl/neJdjUGmwyhPCY6Olc4kV/eu3Aue3Gux3Jno MT8g== X-Gm-Message-State: AOJu0YzANEc56bODSAnHN4ui7UyIMnYjnRgGYQCH+FRACZnNnl7YOaNX wKcsmAw+CV1J3FIM/4Q+OCEpQrza701b8HKJKMmhgwCKpgzvS2/WSV0Y1tA0vp1kDNbm4nlNxhc = X-Gm-Gg: ASbGncujMk/o0Clc25IAc3o2AET1lBvmFsBm2Tku+hqQmLv7AFymim2wAvdVGM5zgiI k64laXpSxApnUCDhNpUQXR1fHptQ4ddYalt6tH6HqvJWbGMAENB6aMb7LZjqWJgY1VmduJwhtEB 3XG7cwE9rSts4uxgNi43ypEaAP/LqzRGzuXw+etZsbOX0nBN1B3Tns/yKu88MtZ+mT8NYFciDvg qZ58ApGtSj7VMCl7LnLC5St7+NsKJuLw9ToDhUZMDVTxlTtNym3swq6etSEXjv6YSgkPvs9Clyy JHRlDOH+Vs+qmCIVdJqyEF55zKOr8NBTibAUobgL/zTczUu2DZawyPUpfSc6 X-Google-Smtp-Source: AGHT+IEz3EqGurLMOBF1f3prn9cEG8crqYMhj6EyIJLXA/MTzAtTS3anMPEQGNXFhmgTaiMI/ffGfg== X-Received: by 2002:a17:90b:4a:b0:2ee:b4d4:69 with SMTP id 98e67ed59e1d1-2efcf2c587fmr7783592a91.35.1733840769318; Tue, 10 Dec 2024 06:26:09 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:08 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v4 5/7] android: Drop notify CAMERA3_MSG_ERROR_REQUEST when a request fails Date: Tue, 10 Dec 2024 14:23:58 +0000 Message-ID: <20241210142557.2886315-6-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" According to Android Camera API v3.2, CAMERA3_MSG_ERROR_REQUEST is used for requests that have not done any processing. When a request is completed with failure, CAMERA3_MSG_ERROR_RESULT should be used instead. To avoid code duplication, when CameraMetadata cannot be generated, CAMERA3_MSG_ERROR_RESULT is notified after process_capture_result. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 40 ++++++++++++++++++++++++++++------- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f3f570544..3b10f207e 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1182,6 +1182,18 @@ void CameraDevice::requestComplete(Request *request) * post-processing/compression fails. */ for (auto &buffer : descriptor->buffers_) { + for (auto &[_, frameBuffer] : request->buffers()) { + if (buffer.srcBuffer != frameBuffer && + buffer.frameBuffer.get() != frameBuffer) + continue; + + StreamBuffer::Status status = StreamBuffer::Status::Success; + if (frameBuffer->metadata().status != FrameMetadata::FrameSuccess) { + status = StreamBuffer::Status::Error; + } + setBufferStatus(buffer, status); + } + CameraStream *stream = buffer.stream; /* @@ -1198,22 +1210,18 @@ void CameraDevice::requestComplete(Request *request) if (fence) buffer.fence = fence->release(); } - buffer.status = StreamBuffer::Status::Success; } /* - * If the Request has failed, abort the request by notifying the error - * and complete the request with all buffers in error state. + * If the Request has failed, complete the request with all buffers in + * error state and notify an error result. */ if (request->status() != Request::RequestComplete) { LOG(HAL, Error) << "Request " << request->cookie() << " not successfully completed: " << request->status(); - abortRequest(descriptor); - completeDescriptor(descriptor); - - return; + descriptor->status_ = Camera3RequestDescriptor::Status::Error; } /* @@ -1238,7 +1246,7 @@ void CameraDevice::requestComplete(Request *request) */ descriptor->resultMetadata_ = getResultMetadata(*descriptor); if (!descriptor->resultMetadata_) { - notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_RESULT); + descriptor->status_ = Camera3RequestDescriptor::Status::Error; /* * The camera framework expects an empty metadata pack on error. @@ -1271,7 +1279,13 @@ void CameraDevice::requestComplete(Request *request) continue; } + if (buffer->status == StreamBuffer::Status::Error) { + iter = descriptor->pendingStreamsToProcess_.erase(iter); + continue; + } + ++iter; + int ret = stream->process(buffer); if (ret) { setBufferStatus(*buffer, StreamBuffer::Status::Error); @@ -1324,6 +1338,16 @@ void CameraDevice::sendCaptureResults() descriptors_.pop(); sendCaptureResult(descriptor.get()); + + /* + * Call notify with CAMERA3_MSG_ERROR_RESULT to indicate some + * of the expected result metadata might not be available + * because the capture is cancelled by the camera. Only notify + * it when the final result is sent, since Android will ignore + * the following metadata. + */ + if (descriptor->status_ == Camera3RequestDescriptor::Status::Error) + notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_RESULT); } } From patchwork Tue Dec 10 14:23:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22274 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 12366C3257 for ; Tue, 10 Dec 2024 14:26:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B9C1D67E87; Tue, 10 Dec 2024 15:26:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SsSyDbz4"; dkim-atps=neutral Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F0A0567E8E for ; Tue, 10 Dec 2024 15:26:12 +0100 (CET) Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-7fd2ff40782so3410448a12.2 for ; Tue, 10 Dec 2024 06:26:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840771; x=1734445571; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FLTZqIt8KEFAnYaGSx3c4gSKC7oTYw6EkdvadMeeIRk=; b=SsSyDbz4WDublwo8jHRNZ1Y5YaGni4e7t1lDVihO/s02xKr90Iz3mXj678Xkq/xppJ hb1p3zbuEVaE4NQtODVkohvlBelCZGDCcNiB4XqTunLu0MdaQTAprnYwCW9mZ0g0yUi2 ukc9cLEocK1sB5AAROXEyzA+077tvTXqnEoxU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840771; x=1734445571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FLTZqIt8KEFAnYaGSx3c4gSKC7oTYw6EkdvadMeeIRk=; b=eIGbtPMcybzWFaia4+wm5euPFYvoW+Vvjp2wYGHlCmOOxfThK9Rn8w1fo7LfVr6q8+ bF+yM5GBtlpoLzrLNET0V6EAxtNIl0N7DgNPlPhBELHQwv4QDeXkhr8lex9d6clH/Qs8 zXyXlZ6Ij+TTZ4D0Mx4T/eiYlP9kRBAgGiUn6TJ66DwNXjky0A9U/rBqRAWzb8vjduDS RzgmJ0+8ahHnkrCErRWVfMdjor5jsMC3kEKMQNaY7/SOZcNBRBD7PhMVVf2jqh6oJN3L 1Tyx9fcvQQSqYJrPR08P3kIVcVJF/E8c+httGgZT4ozqKPVtJCOE2p2DK9LRLEVnnAhD 99PA== X-Gm-Message-State: AOJu0YwH+EnSjbqsGRDqKRItfxCqOKseudBh0zdKrvy0z7CA+7EgxDEe p/ooj8Zn/EzPbCN4SJn0RtjqOd7//vmm4ijYhxKm9FRg0q/0uoYe0ffbA38n+sKvKJaKa1qOnTU = X-Gm-Gg: ASbGnctIrcD4nN9p5rSMOPoHhKgQkB7kZOfgPmx19q2DyQrVwUNbWJPa8c4wu6Cog9d xX4ofEDNBgReAaRWhS0EtQogXQPB+Vt2TPJ0DCKQM315xd2g7kUR30HZSGMml6bpW0qLnY2+A7Q tTU+KdszHmYw71bcMZWhRt0vCPmNWnqRoN8fOuT+jSZ7UEdpM0nf+L8hHOpf/WIFz1pYQhg0Z4l wFjGxKlEYoZ0L6K/Y4Ylm42vUWN7zcZW2SRoG2tkOiE9U7xaVQzGLhS6DFgY0kHcS6AwioQMFic 3qx6FeTNVNNYXZKkpSQXgm8RziUjJotXPE66ti7OBTsPVE45Y/qvlydBOftB X-Google-Smtp-Source: AGHT+IF6M579DIVfStKTVeNnIJUYHLFxMaFwGSkzzttNyJUMjFutqo+MY0vju63/d+qq5M2MDo0Vww== X-Received: by 2002:a17:90b:5111:b0:2ee:ab29:1482 with SMTP id 98e67ed59e1d1-2efcf18d802mr7692187a91.16.1733840770955; Tue, 10 Dec 2024 06:26:10 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:10 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen , Jacopo Mondi Subject: [PATCH v4 6/7] android: notify CAMERA3_MSG_ERROR_REQUEST out of order Date: Tue, 10 Dec 2024 14:23:59 +0000 Message-ID: <20241210142557.2886315-7-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request hasn't done any processing, CAMERA3_MSG_ERROR_REQUEST and the following process_capture_result don't need to wait for the previous requests' completion. This patch avoids pushing the aborted requests into the request queue. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi --- src/android/camera_device.cpp | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 3b10f207e..a95114c8d 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -867,6 +867,8 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const buffer.status = StreamBuffer::Status::Error; descriptor->status_ = Camera3RequestDescriptor::Status::Error; + + sendCaptureResult(descriptor); } bool CameraDevice::isValidRequest(camera3_capture_request_t *camera3Request) const @@ -1135,14 +1137,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques MutexLocker stateLock(stateMutex_); if (state_ == State::Flushing) { - Camera3RequestDescriptor *rawDescriptor = descriptor.get(); - { - MutexLocker descriptorsLock(descriptorsMutex_); - descriptors_.push(std::move(descriptor)); - } - abortRequest(rawDescriptor); - completeDescriptor(rawDescriptor); - + abortRequest(descriptor.get()); return 0; } From patchwork Tue Dec 10 14:24:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22275 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD16FC32DC for ; Tue, 10 Dec 2024 14:26:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 81B4A67E96; Tue, 10 Dec 2024 15:26:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="QemRdkVS"; dkim-atps=neutral Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4618C67E88 for ; Tue, 10 Dec 2024 15:26:14 +0100 (CET) Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-2ee51f8c47dso4136362a91.1 for ; Tue, 10 Dec 2024 06:26:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1733840772; x=1734445572; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QsaKE+pnPxyxFfHOwhaGtd2pcOwdYEqO6rU+o+44mQk=; b=QemRdkVSxr8G9kcjQQ7KWIeHiUlwAuEinwBmJfJ/ItRzGsUmF9NADlljaquNjwF1g7 v5bWk0Yckfad3NYUDx+x5TqaIwWYCA7qcnXXNbtgtB8UT/S+PnOBUc4iCdLiRKqwuGAd IoHjAPxC+LaEGuViKrziyoEWMaxzM2E6HelGw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733840772; x=1734445572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QsaKE+pnPxyxFfHOwhaGtd2pcOwdYEqO6rU+o+44mQk=; b=awI1UodWgNSUifOEAuLwgcFx+keEKUi9cP9cKzCkzgz0162+jxp04flYIikzl+mie8 QjsP6FetK41aathUOV9HDk1bLb58Ncve6u7YGZpNUgPkkZPqSGf5B4ou/1FsPPVNTn9Z wS46wZu0ZxGFTmSdN+UvbbNaJNCX+mMTZyDVResbhzEKw3nKs3bukV1kDFzbIt0tW5o+ /Fc0c/raYeYMZ5sprdiIcMnmHKa8l6M2ekxswJL592K6JlkFufg6hoMz2RGgs4bt46oI 9/0i2Ghytho8XNmRIEZEoom3zZcxxu7j1jW231SA0yJiyT3HjTc2s3O3dMi5HWNOUXRN pJkg== X-Gm-Message-State: AOJu0YzcTttrTDPLj3E7dWYQjVsw7+HTp0SKCzosQ9sgNqH91DyeHwWk 2KFhZtEfULs6l0CR9HXXsyQculZKFYtv4p0thjbWMZI4+3tuxp8Mc0uaPa0SNFN9rqzhNbz7s4k = X-Gm-Gg: ASbGncv+NDrjZ8OMiyH+7RvK5JE7aWIWI/PFMoMMPiAtV/j52C04Pc+Bs65XxsL/miS nH4N101vI+cm39lyw+a2MghXEEI2qtk4zEWU6y2eP+3LkNAzQ3Yvok0A591rdr0KO7JuLtzJ8Wa PEgAHVVJWvI8rCy3M1wsorVYRnUZzv0pdNfBUCdxsTf1WpTctXPmvPW3eFgW2k085efxdLEJztT bZLzd3Ka/92ImvL2IzOAAUtfEC26Oqt0wG1n7Ny0B7DP1i600yG9o+jZ+Mf+OCMDGi+h4Hq9XH/ mgnquPBefsn+0Ldtfm6h0NLaaiuC9Z8AVCbXCLFfuh8HYDnuTyMVQXWeuBuS X-Google-Smtp-Source: AGHT+IE3YA0vbFyY+16obTdgKt7fZmBsAyX0Dut7K7I8LkoLktIsyBQnza30udAzvhms3Juelm7GCw== X-Received: by 2002:a17:90b:35d2:b0:2ee:f80c:6889 with SMTP id 98e67ed59e1d1-2efcf26e386mr7414361a91.33.1733840772468; Tue, 10 Dec 2024 06:26:12 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (112.212.229.35.bc.googleusercontent.com. [35.229.212.112]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97f2sm11812053a91.11.2024.12.10.06.26.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 06:26:12 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Han-Lin Chen Subject: [PATCH v4 7/7] android: Remove Camera3RequestDescriptor::streamsProcessMutex_ Date: Tue, 10 Dec 2024 14:24:00 +0000 Message-ID: <20241210142557.2886315-8-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241210142557.2886315-1-chenghaoyang@chromium.org> References: <20241210142557.2886315-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This mutex was needed when CameraStream's worker thread posts a result back to CameraDevice. We can simplify it by calling CameraDevice's function on libcamera::Camera's owner thread. With this delegation, `Camera3RequestDescriptor::pendingStreamsToProcess_` will be firstly setup in the application's thread in processCaptureRequest(), and the rest accesses will be done in libcamera::CameraManager's thread. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 33 +++++++++++++++++---------------- src/android/camera_device.h | 6 ++++-- src/android/camera_request.h | 4 +--- src/android/camera_stream.cpp | 4 ++-- 4 files changed, 24 insertions(+), 23 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a95114c8d..cb819938a 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -989,8 +989,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques FrameBuffer *frameBuffer = nullptr; UniqueFD acquireFence; - MutexLocker lock(descriptor->streamsProcessMutex_); - switch (cameraStream->type()) { case CameraStream::Type::Mapped: /* Mapped streams will be handled in the next loop. */ @@ -1066,7 +1064,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques << cameraStream->configuration().pixelFormat << "]" << " (mapped)"; - MutexLocker lock(descriptor->streamsProcessMutex_); descriptor->pendingStreamsToProcess_.insert({ cameraStream, &buffer }); /* @@ -1252,9 +1249,6 @@ void CameraDevice::requestComplete(Request *request) descriptor->resultMetadata_ = std::make_unique(0, 0); } - /* Handle post-processing. */ - MutexLocker locker(descriptor->streamsProcessMutex_); - /* * Queue all the post-processing streams request at once. The completion * slot streamProcessingComplete() can only execute when we are out @@ -1288,10 +1282,8 @@ void CameraDevice::requestComplete(Request *request) } } - if (descriptor->pendingStreamsToProcess_.empty()) { - locker.unlock(); + if (descriptor->pendingStreamsToProcess_.empty()) completeDescriptor(descriptor); - } } /** @@ -1398,6 +1390,19 @@ void CameraDevice::setBufferStatus(StreamBuffer &streamBuffer, } } +void CameraDevice::streamProcessingCompleteDelegate(StreamBuffer *streamBuffer, + StreamBuffer::Status status) +{ + /* + * Delegate the callback to the camera manager thread to simplify race condition. + */ + auto *method = new BoundMethodMember{ + this, camera_.get(), &CameraDevice::streamProcessingComplete, ConnectionTypeQueued + }; + + method->activate(streamBuffer, status); +} + /** * \brief Handle post-processing completion of a stream in a capture request * \param[in] streamBuffer The StreamBuffer for which processing is complete @@ -1418,13 +1423,9 @@ void CameraDevice::streamProcessingComplete(StreamBuffer *streamBuffer, Camera3RequestDescriptor *request = streamBuffer->request; - { - MutexLocker locker(request->streamsProcessMutex_); - - request->pendingStreamsToProcess_.erase(streamBuffer->stream); - if (!request->pendingStreamsToProcess_.empty()) - return; - } + request->pendingStreamsToProcess_.erase(streamBuffer->stream); + if (!request->pendingStreamsToProcess_.empty()) + return; completeDescriptor(streamBuffer->request); } diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 699aa8f17..69d163d76 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -65,8 +65,8 @@ public: int configureStreams(camera3_stream_configuration_t *stream_list); int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); - void streamProcessingComplete(StreamBuffer *bufferStream, - StreamBuffer::Status status); + void streamProcessingCompleteDelegate(StreamBuffer *bufferStream, + StreamBuffer::Status status); protected: std::string logPrefix() const override; @@ -96,6 +96,8 @@ private: int processControls(Camera3RequestDescriptor *descriptor); void completeDescriptor(Camera3RequestDescriptor *descriptor) LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); + void streamProcessingComplete(StreamBuffer *bufferStream, + StreamBuffer::Status status); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); void sendCaptureResult(Camera3RequestDescriptor *request) const; void setBufferStatus(StreamBuffer &buffer, diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 6b2a00795..bd75d4595 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -66,9 +66,7 @@ public: }; /* Keeps track of streams requiring post-processing. */ - std::map pendingStreamsToProcess_ - LIBCAMERA_TSA_GUARDED_BY(streamsProcessMutex_); - libcamera::Mutex streamsProcessMutex_; + std::map pendingStreamsToProcess_; Camera3RequestDescriptor(libcamera::Camera *camera, const camera3_capture_request_t *camera3Request); diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 53f292d4b..7837fd7aa 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -121,8 +121,8 @@ int CameraStream::configure() else bufferStatus = StreamBuffer::Status::Error; - cameraDevice_->streamProcessingComplete(streamBuffer, - bufferStatus); + cameraDevice_->streamProcessingCompleteDelegate(streamBuffer, + bufferStatus); }); worker_->start();