From patchwork Fri Apr 2 02:12:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11822 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF629BD695 for ; Fri, 2 Apr 2021 02:12:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 329D568780; Fri, 2 Apr 2021 04:12:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="aOoWuYX8"; dkim-atps=neutral Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C4A268780 for ; Fri, 2 Apr 2021 04:12:50 +0200 (CEST) Received: by mail-pl1-x629.google.com with SMTP id f17so1931058plr.0 for ; Thu, 01 Apr 2021 19:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JCSQ+v9lhJGtYTCD7hMrSkJDED8YDPSBgNcSanoVgks=; b=aOoWuYX8UD2ABfbrFV0FfsnNor9EDp20LRSsUzDT+tqdCucwM56cCo3hNY9ZivBITm kaZ92YfrYnewI+gwYlH2YKyW3WRSAM7KUhGjYJISNbJPSGcGs4VPmCJnXS8nXN7BsYhN v9mNN0bCmMqKo+5h2ReARx9DFWr4qZfzSRlpk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JCSQ+v9lhJGtYTCD7hMrSkJDED8YDPSBgNcSanoVgks=; b=XeqMKUA1HKIR6PanWmAFS5arMXTXHvaT3WmR+Zw8l9Olmap68oJuFEeFM312Q6Mbom DKxOe8NqEF8z/lu1PG6gRkwx0mdMwxgX8QNLi6TyFxXJE9lwzLH+nIjlqkcia+u6dYAj bU9PiYRxbs1TJxTiJ1+FufRpSJkIThJENXHYRokTcXnZ5edXInIJ1OBWrlCari2PfG0f +kxOJRxb5hV2RrrhA6VjfDu4wEpdeQyH5NQJNAvAdtwmL7I/J9XkkwhxeTRT2beXhzBD seKO5hRsprd/MXl5taDBFPcy34Egaytz0B69oPzXXm87i+gssqPShe88UcPIKQvJLbBy dkcg== X-Gm-Message-State: AOAM533uyISxO2Eqs+YxoRvks/7dS5pVd40h8o6IZQwTnnvr1bruneHO t4OkWA06abSqQU5+iYeMy1A4oXrQcSkTgQ== X-Google-Smtp-Source: ABdhPJwefR5VvlxTxtPXG6uEIOVwDXz4sJWr2Uoo0mXkez6VgjoIlvqYxttR0Xpk8i7vaHrKDzfShw== X-Received: by 2002:a17:903:10d:b029:e7:3875:33ed with SMTP id y13-20020a170903010db02900e7387533edmr10791120plc.34.1617329568320; Thu, 01 Apr 2021 19:12:48 -0700 (PDT) Received: from hiroh2.tok.corp.google.com ([2401:fa00:8f:2:908:1da:b07c:32bc]) by smtp.gmail.com with ESMTPSA id p4sm86902pjk.57.2021.04.01.19.12.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 19:12:47 -0700 (PDT) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Apr 2021 11:12:38 +0900 Message-Id: <20210402021238.1297591-3-hiroh@chromium.org> X-Mailer: git-send-email 2.31.0.208.g409f899ff0-goog In-Reply-To: <20210402021238.1297591-1-hiroh@chromium.org> References: <20210402021238.1297591-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/2] android: CameraDevice: Fix Camera3RequestDescriptor leakage X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" CameraDevice creates Camera3RequestDescriptor in processCaptureRequest() and disallocates in requestComplete(). Camera3RequestDescriptor can never be destroyed if requestComplete() is never called. This avoid the memory leakage by storing them in map CameraRequestDescriptor. Signed-off-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- src/android/camera_device.cpp | 83 +++++++++++++++++++++-------------- src/android/camera_device.h | 11 +++-- src/android/camera_worker.cpp | 4 +- src/android/camera_worker.h | 2 +- 4 files changed, 60 insertions(+), 40 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index b45d3a54..3d98e54c 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -9,9 +9,11 @@ #include "camera_ops.h" #include "post_processor.h" +#include #include #include #include +#include #include #include @@ -287,15 +289,11 @@ CameraDevice::Camera3RequestDescriptor::Camera3RequestDescriptor( /* * Create the libcamera::Request unique_ptr<> to tie its lifetime - * to the descriptor's one. Set the descriptor's address as the - * request's cookie to retrieve it at completion time. + * to the descriptor's one. */ - request_ = std::make_unique(camera, - reinterpret_cast(this)); + request_ = std::make_unique(camera); } -CameraDevice::Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; - /* * \class CameraDevice * @@ -672,6 +670,7 @@ void CameraDevice::stop() worker_.stop(); camera_->stop(); + descriptors_.clear(); running_ = false; } @@ -1818,8 +1817,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * The descriptor and the associated memory reserved here are freed * at request complete time. */ - Camera3RequestDescriptor *descriptor = - new Camera3RequestDescriptor(camera_.get(), camera3Request); + Camera3RequestDescriptor descriptor(camera_.get(), camera3Request); + /* * \todo The Android request model is incremental, settings passed in * previous requests are to be effective until overridden explicitly in @@ -1829,12 +1828,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques if (camera3Request->settings) lastSettings_ = camera3Request->settings; else - descriptor->settings_ = lastSettings_; + descriptor.settings_ = lastSettings_; - LOG(HAL, Debug) << "Queueing request " << descriptor->request_->cookie() - << " with " << descriptor->buffers_.size() << " streams"; - for (unsigned int i = 0; i < descriptor->buffers_.size(); ++i) { - const camera3_stream_buffer_t &camera3Buffer = descriptor->buffers_[i]; + LOG(HAL, Debug) << "Queueing request " << descriptor.request_->cookie() + << " with " << descriptor.buffers_.size() << " streams"; + for (unsigned int i = 0; i < descriptor.buffers_.size(); ++i) { + const camera3_stream_buffer_t &camera3Buffer = descriptor.buffers_[i]; camera3_stream *camera3Stream = camera3Buffer.stream; CameraStream *cameraStream = static_cast(camera3Stream->priv); @@ -1867,7 +1866,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * lifetime management only. */ buffer = createFrameBuffer(*camera3Buffer.buffer); - descriptor->frameBuffers_.emplace_back(buffer); + descriptor.frameBuffers_.emplace_back(buffer); LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -1886,11 +1885,10 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques if (!buffer) { LOG(HAL, Error) << "Failed to create buffer"; - delete descriptor; return -ENOMEM; } - descriptor->request_->addBuffer(cameraStream->stream(), buffer, + descriptor.request_->addBuffer(cameraStream->stream(), buffer, camera3Buffer.acquire_fence); } @@ -1898,11 +1896,16 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * Translate controls from Android to libcamera and queue the request * to the CameraWorker thread. */ - int ret = processControls(descriptor); + int ret = processControls(&descriptor); if (ret) return ret; - worker_.queueRequest(descriptor->request_.get()); + worker_.queueRequest(descriptor.request_.get()); + + { + std::scoped_lock lock(mutex_); + descriptors_[descriptor.request_->cookie()] = std::move(descriptor); + } return 0; } @@ -1912,8 +1915,18 @@ void CameraDevice::requestComplete(Request *request) const Request::BufferMap &buffers = request->buffers(); camera3_buffer_status status = CAMERA3_BUFFER_STATUS_OK; std::unique_ptr resultMetadata; - Camera3RequestDescriptor *descriptor = - reinterpret_cast(request->cookie()); + + Camera3RequestDescriptor descriptor; + { + std::scoped_lock lock(mutex_); + auto it = descriptors_.find(request->cookie()); + if (it == descriptors_.end()) { + LOG(HAL, Error) << "Unknown request: " << request->cookie(); + status = CAMERA3_BUFFER_STATUS_ERROR; + return; + } + descriptor = std::move(it->second); + } if (request->status() != Request::RequestComplete) { LOG(HAL, Error) << "Request not successfully completed: " @@ -1922,7 +1935,7 @@ void CameraDevice::requestComplete(Request *request) } LOG(HAL, Debug) << "Request " << request->cookie() << " completed with " - << descriptor->buffers_.size() << " streams"; + << descriptor.buffers_.size() << " streams"; /* * \todo The timestamp used for the metadata is currently always taken @@ -1931,10 +1944,10 @@ void CameraDevice::requestComplete(Request *request) * pipeline handlers) timestamp in the Request itself. */ uint64_t timestamp = buffers.begin()->second->metadata().timestamp; - resultMetadata = getResultMetadata(*descriptor, timestamp); + resultMetadata = getResultMetadata(descriptor, timestamp); /* Handle any JPEG compression. */ - for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { + for (camera3_stream_buffer_t &buffer : descriptor.buffers_) { CameraStream *cameraStream = static_cast(buffer.stream->priv); @@ -1949,7 +1962,7 @@ void CameraDevice::requestComplete(Request *request) int ret = cameraStream->process(*src, *buffer.buffer, - descriptor->settings_, + descriptor.settings_, resultMetadata.get()); if (ret) { status = CAMERA3_BUFFER_STATUS_ERROR; @@ -1966,17 +1979,17 @@ void CameraDevice::requestComplete(Request *request) /* Prepare to call back the Android camera stack. */ camera3_capture_result_t captureResult = {}; - captureResult.frame_number = descriptor->frameNumber_; - captureResult.num_output_buffers = descriptor->buffers_.size(); - for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { + captureResult.frame_number = descriptor.frameNumber_; + captureResult.num_output_buffers = descriptor.buffers_.size(); + for (camera3_stream_buffer_t &buffer : descriptor.buffers_) { buffer.acquire_fence = -1; buffer.release_fence = -1; buffer.status = status; } - captureResult.output_buffers = descriptor->buffers_.data(); + captureResult.output_buffers = descriptor.buffers_.data(); if (status == CAMERA3_BUFFER_STATUS_OK) { - notifyShutter(descriptor->frameNumber_, timestamp); + notifyShutter(descriptor.frameNumber_, timestamp); captureResult.partial_result = 1; captureResult.result = resultMetadata->get(); @@ -1989,13 +2002,15 @@ void CameraDevice::requestComplete(Request *request) * is here signalled. Make sure the error path plays well with * the camera stack state machine. */ - notifyError(descriptor->frameNumber_, - descriptor->buffers_[0].stream); + notifyError(descriptor.frameNumber_, + descriptor.buffers_[0].stream); } - callbacks_->process_capture_result(callbacks_, &captureResult); - - delete descriptor; + { + std::scoped_lock lock(mutex_); + descriptors_.erase(request->cookie()); + callbacks_->process_capture_result(callbacks_, &captureResult); + } } std::string CameraDevice::logPrefix() const diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 39cf95ad..e4ee0cb0 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -61,7 +62,6 @@ public: int configureStreams(camera3_stream_configuration_t *stream_list); int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); - protected: std::string logPrefix() const override; @@ -69,11 +69,13 @@ private: CameraDevice(unsigned int id, std::shared_ptr camera); struct Camera3RequestDescriptor { + Camera3RequestDescriptor() = default; + ~Camera3RequestDescriptor() = default; Camera3RequestDescriptor(libcamera::Camera *camera, const camera3_capture_request_t *camera3Request); - ~Camera3RequestDescriptor(); + Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; - uint32_t frameNumber_; + uint32_t frameNumber_ = 0; std::vector buffers_; std::vector> frameBuffers_; CameraMetadata settings_; @@ -124,6 +126,9 @@ private: std::map formatsMap_; std::vector streams_; + std::mutex mutex_; + std::map descriptors_; + std::string maker_; std::string model_; diff --git a/src/android/camera_worker.cpp b/src/android/camera_worker.cpp index df436460..300ddde0 100644 --- a/src/android/camera_worker.cpp +++ b/src/android/camera_worker.cpp @@ -26,10 +26,10 @@ LOG_DECLARE_CATEGORY(HAL) * by the CameraWorker which queues it to the libcamera::Camera after handling * fences. */ -CaptureRequest::CaptureRequest(libcamera::Camera *camera, uint64_t cookie) +CaptureRequest::CaptureRequest(libcamera::Camera *camera) : camera_(camera) { - request_ = camera_->createRequest(cookie); + request_ = camera_->createRequest(reinterpret_cast(this)); } void CaptureRequest::addBuffer(Stream *stream, FrameBuffer *buffer, int fence) diff --git a/src/android/camera_worker.h b/src/android/camera_worker.h index d7060576..64b1658b 100644 --- a/src/android/camera_worker.h +++ b/src/android/camera_worker.h @@ -22,7 +22,7 @@ class CameraDevice; class CaptureRequest { public: - CaptureRequest(libcamera::Camera *camera, uint64_t cookie); + CaptureRequest(libcamera::Camera *camera); const std::vector &fences() const { return acquireFences_; } libcamera::ControlList &controls() { return request_->controls(); }