From patchwork Sat Mar 18 23:40:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 18415 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B854CC329F for ; Sat, 18 Mar 2023 23:40:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 506B262742; Sun, 19 Mar 2023 00:40:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679182839; bh=qGIT2CBv0+7CHNkyZokcI9Z9R0gjeu37xWvNHSRIRXQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=bPSuoyjNiJaESwvTlLm4Sowb8euT+yK/CIVA8cjS0VKHjpul6DYYt+DMv6FwL/Zmm vHTMgvJLpqthpteonPNRpWAcoWwlQ+FVSrXTd/g8/LrN5P0EkQHdq40llu456Z0gop f7LKzvBYsIIkYBYczENUG2Z1eRaSt8nMXeE3IqgqI4laoYlq69LZVpChnKIpwcdwIy 1efwqKzfhSyV44OhKqPW/JmZZKTs1zPPRSJSqLXnVKDSROUsQ/MHQFHi1A3HdwZ1z7 qssn6u1en3GblmzhaI/+tQDpL4W6ETStpxaQlINVW234bJRFNc/Rz9pNRvB1mztyGP pHrhQO6Byheng== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B730C626EC for ; Sun, 19 Mar 2023 00:40:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F/5tjym8"; dkim-atps=neutral Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 53832DF7; Sun, 19 Mar 2023 00:40:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1679182833; bh=qGIT2CBv0+7CHNkyZokcI9Z9R0gjeu37xWvNHSRIRXQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F/5tjym8zV8rrhVMN2cOWvEoKTlh5Itu2x/PHnQiR/mkQ6GLgZ1nlYTXViqDYa6uR nJsNMwEWoRfLhy6RS39JHUtaOWh/OEBYKmbolAdMrvaoXUl+ybVG7HVCkCurvQz1Sm tBXN4MTKBMNOMoPu7rtRPj7LRXqhJdadY51u9RrU= To: libcamera-devel@lists.libcamera.org Date: Sat, 18 Mar 2023 23:40:09 +0000 Message-Id: <20230318234014.29506-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230318234014.29506-1-dan.scally@ideasonboard.com> References: <20230318234014.29506-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/11] pipeline: ipu3: Support sensors using only CIO2 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Daniel Scally via libcamera-devel From: Dan Scally Reply-To: Daniel Scally Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some sensors that need to be supported by the IPU3 pipeline are not supported by the Imgu's kernel driver, but can be used by simply treating the CIO2's capture device as the source of image data. In that case much of the processing in the IPU3 pipeline needs to be disabled. Guard it behind checks that query the CIO2Device::needsImgu field. Signed-off-by: Daniel Scally --- src/libcamera/pipeline/ipu3/ipu3.cpp | 214 +++++++++++++++------------ 1 file changed, 118 insertions(+), 96 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 3077c6bb..cc7da299 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -489,6 +489,34 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) V4L2DeviceFormat outputFormat; int ret; + /* + * Pass the requested stream size to the CIO2 unit and get back the + * adjusted format to be propagated to the ImgU output devices. + */ + const Size &sensorSize = config->cio2Format().size; + V4L2DeviceFormat cio2Format; + ret = cio2->configure(sensorSize, config->combinedTransform_, &cio2Format); + if (ret) + return ret; + + IPACameraSensorInfo sensorInfo; + cio2->sensor()->sensorInfo(&sensorInfo); + data->cropRegion_ = sensorInfo.analogCrop; + + /* + * If the ImgU gets configured, its driver seems to expect that + * buffers will be queued to its outputs, as otherwise the next + * capture session that uses the ImgU fails when queueing + * buffers to its input. + * + * If no ImgU configuration has been computed, it means only a RAW + * stream has been requested: return here to skip the ImgU configuration + * part. + */ + ImgUDevice::PipeConfig imguConfig = config->imguConfig(); + if (imguConfig.isNull()) + return 0; + /* * FIXME: enabled links in one ImgU pipe interfere with capture * operations on the other one. This can be easily triggered by @@ -531,34 +559,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; - /* - * Pass the requested stream size to the CIO2 unit and get back the - * adjusted format to be propagated to the ImgU output devices. - */ - const Size &sensorSize = config->cio2Format().size; - V4L2DeviceFormat cio2Format; - ret = cio2->configure(sensorSize, config->combinedTransform_, &cio2Format); - if (ret) - return ret; - - IPACameraSensorInfo sensorInfo; - cio2->sensor()->sensorInfo(&sensorInfo); - data->cropRegion_ = sensorInfo.analogCrop; - - /* - * If the ImgU gets configured, its driver seems to expect that - * buffers will be queued to its outputs, as otherwise the next - * capture session that uses the ImgU fails when queueing - * buffers to its input. - * - * If no ImgU configuration has been computed, it means only a RAW - * stream has been requested: return here to skip the ImgU configuration - * part. - */ - ImgUDevice::PipeConfig imguConfig = config->imguConfig(); - if (imguConfig.isNull()) - return 0; - ret = imgu->configure(imguConfig, &cio2Format); if (ret) return ret; @@ -660,36 +660,39 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, int PipelineHandlerIPU3::allocateBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); + CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; unsigned int bufferCount; int ret; - bufferCount = std::max({ - data->outStream_.configuration().bufferCount, - data->vfStream_.configuration().bufferCount, - data->rawStream_.configuration().bufferCount, - }); + if (cio2->needsImgu()) { + bufferCount = std::max({ + data->outStream_.configuration().bufferCount, + data->vfStream_.configuration().bufferCount, + data->rawStream_.configuration().bufferCount, + }); - ret = imgu->allocateBuffers(bufferCount); - if (ret < 0) - return ret; + ret = imgu->allocateBuffers(bufferCount); + if (ret < 0) + return ret; - /* Map buffers to the IPA. */ - unsigned int ipaBufferId = 1; + /* Map buffers to the IPA. */ + unsigned int ipaBufferId = 1; - for (const std::unique_ptr &buffer : imgu->paramBuffers_) { - buffer->setCookie(ipaBufferId++); - ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); - } + for (const std::unique_ptr &buffer : imgu->paramBuffers_) { + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + } - for (const std::unique_ptr &buffer : imgu->statBuffers_) { - buffer->setCookie(ipaBufferId++); - ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); - } + for (const std::unique_ptr &buffer : imgu->statBuffers_) { + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + } - data->ipa_->mapBuffers(ipaBuffers_); + data->ipa_->mapBuffers(ipaBuffers_); + data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); + } - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -699,17 +702,20 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) int PipelineHandlerIPU3::freeBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); + CIO2Device *cio2 = &data->cio2_; data->frameInfos_.clear(); - std::vector ids; - for (IPABuffer &ipabuf : ipaBuffers_) - ids.push_back(ipabuf.id); + if (cio2->needsImgu()) { + std::vector ids; + for (IPABuffer &ipabuf : ipaBuffers_) + ids.push_back(ipabuf.id); - data->ipa_->unmapBuffers(ids); - ipaBuffers_.clear(); + data->ipa_->unmapBuffers(ids); + data->imgu_->freeBuffers(); - data->imgu_->freeBuffers(); + ipaBuffers_.clear(); + } return 0; } @@ -736,10 +742,6 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) return ret; - ret = data->ipa_->start(); - if (ret) - goto error; - data->delayedCtrls_->reset(); /* @@ -750,16 +752,24 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; - ret = imgu->start(); - if (ret) - goto error; + if (cio2->needsImgu()) { + ret = data->ipa_->start(); + if (ret) + goto error; + + ret = imgu->start(); + if (ret) + goto error; + } return 0; error: - imgu->stop(); cio2->stop(); - data->ipa_->stop(); + if (cio2->needsImgu()) { + imgu->stop(); + data->ipa_->stop(); + } freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); @@ -773,9 +783,11 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->cancelPendingRequests(); - data->ipa_->stop(); + if (data->cio2_.needsImgu()) { + data->ipa_->stop(); + ret |= data->imgu_->stop(); + } - ret |= data->imgu_->stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); @@ -806,7 +818,7 @@ void IPU3CameraData::queuePendingRequests() while (!pendingRequests_.empty()) { Request *request = pendingRequests_.front(); - IPU3Frames::Info *info = frameInfos_.create(request, true); + IPU3Frames::Info *info = frameInfos_.create(request, cio2_.needsImgu()); if (!info) break; @@ -829,7 +841,8 @@ void IPU3CameraData::queuePendingRequests() info->rawBuffer = rawBuffer; - ipa_->queueRequest(info->id, request->controls()); + if (cio2_.needsImgu()) + ipa_->queueRequest(info->id, request->controls()); pendingRequests_.pop(); processingRequests_.push(request); @@ -1070,10 +1083,6 @@ int PipelineHandlerIPU3::registerCameras() if (ret) continue; - ret = data->loadIPA(); - if (ret) - continue; - /* Initialize the camera properties. */ data->properties_ = cio2->sensor()->properties(); @@ -1104,36 +1113,44 @@ int PipelineHandlerIPU3::registerCameras() << cio2->sensor()->id() << ". Assume rotation 0"; - /** - * \todo Dynamically assign ImgU and output devices to each - * stream and camera; as of now, limit support to two cameras - * only, and assign imgu0 to the first one and imgu1 to the - * second. - */ - data->imgu_ = numCameras ? &imgu1_ : &imgu0_; - /* * Connect video devices' 'bufferReady' signals to their * slot to implement the image processing pipeline. - * - * Frames produced by the CIO2 unit are passed to the - * associated ImgU input where they get processed and - * returned through the ImgU main and secondary outputs. */ data->cio2_.bufferReady().connect(data.get(), - &IPU3CameraData::cio2BufferReady); + &IPU3CameraData::cio2BufferReady); data->cio2_.bufferAvailable.connect( data.get(), &IPU3CameraData::queuePendingRequests); - data->imgu_->input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); - data->imgu_->output_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->viewfinder_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->param_->bufferReady.connect(data.get(), - &IPU3CameraData::paramBufferReady); - data->imgu_->stat_->bufferReady.connect(data.get(), - &IPU3CameraData::statBufferReady); + + if (cio2->needsImgu()) { + ret = data->loadIPA(); + if (ret) + continue; + + /** + * \todo Dynamically assign ImgU and output devices to each + * stream and camera; as of now, limit support to two cameras + * only, and assign imgu0 to the first one and imgu1 to the + * second. + */ + data->imgu_ = numCameras ? &imgu1_ : &imgu0_; + + /* + * Frames produced by the CIO2 unit are passed to the + * associated ImgU input where they get processed and + * returned through the ImgU main and secondary outputs. + */ + data->imgu_->input_->bufferReady.connect(&data->cio2_, + &CIO2Device::tryReturnBuffer); + data->imgu_->output_->bufferReady.connect(data.get(), + &IPU3CameraData::imguOutputBufferReady); + data->imgu_->viewfinder_->bufferReady.connect(data.get(), + &IPU3CameraData::imguOutputBufferReady); + data->imgu_->param_->bufferReady.connect(data.get(), + &IPU3CameraData::paramBufferReady); + data->imgu_->stat_->bufferReady.connect(data.get(), + &IPU3CameraData::statBufferReady); + } /* Create and register the Camera instance. */ const std::string &cameraId = cio2->sensor()->id(); @@ -1330,10 +1347,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); - if (request->findBuffer(&rawStream_)) + if (request->findBuffer(&rawStream_)) { pipe()->completeBuffer(request, buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + if (frameInfos_.tryComplete(info)) + pipe()->completeRequest(request); + } + + if (cio2_.needsImgu()) + ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)