From patchwork Wed Apr 3 15:07:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 906 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D45F60DB2 for ; Wed, 3 Apr 2019 17:06:56 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id B370FFF816; Wed, 3 Apr 2019 15:06:55 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:28 +0200 Message-Id: <20190403150735.27580-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/8] libcamera: utils: Define BIT() macro X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:06:56 -0000 Define BIT(b_) macro which expands to a left bit shift. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/include/utils.h | 1 + src/libcamera/utils.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/src/libcamera/include/utils.h b/src/libcamera/include/utils.h index 79038a96feab..1a6cf7f7b9dc 100644 --- a/src/libcamera/include/utils.h +++ b/src/libcamera/include/utils.h @@ -10,6 +10,7 @@ #include #define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0])) +#define BIT(b_) (1 << (b_)) namespace libcamera { diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp index cd0fd7614cc7..1a5a2a03b1ca 100644 --- a/src/libcamera/utils.cpp +++ b/src/libcamera/utils.cpp @@ -24,6 +24,11 @@ namespace utils { * \brief Determine the number of elements in the static array. */ +/** + * \def BIT(b) + * \brief Bitwise left shift by \a b bits + */ + /** * \brief Strip the directory prefix from the path * \param[in] path The path to process From patchwork Wed Apr 3 15:07:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 907 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B8061610C5 for ; Wed, 3 Apr 2019 17:06:56 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 4F1D7FF806; Wed, 3 Apr 2019 15:06:56 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:29 +0200 Message-Id: <20190403150735.27580-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/8] libcamera: ipu3: Create camera with 2 streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:06:57 -0000 Create each IPU3 camera with two streams: 'output' and 'viewfinder' which represents the video stream from main and secondary ImgU output respectively. Re-work stream configuration to handle the two video streams 'output' and 'viewfinder' separately. As the IPU3 driver requires viewfinder and stat video nodes to be started not to stall ImgU processing, keep track of which streams have been requested by the application, and configure 'output', 'viewfinder' and 'stat' regardless of the user requests. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 216 +++++++++++++++++++++------ 1 file changed, 170 insertions(+), 46 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 164e187c769d..caf1051c58ab 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -171,17 +171,61 @@ private: CIO2Device cio2_; ImgUDevice *imgu_; - Stream stream_; + Stream outStream_; + Stream vfStream_; + + unsigned int activeStreamsMask; }; static constexpr unsigned int IPU3_BUFFER_COUNT = 4; + static constexpr unsigned int IPU3_STREAM_OUTPUT = BIT(0); + static constexpr unsigned int IPU3_STREAM_VF = BIT(1); + IPU3CameraData *cameraData(const Camera *camera) { return static_cast( PipelineHandler::cameraData(camera)); } + bool isOutput(IPU3CameraData *data, Stream *stream) + { + return &data->outStream_ == stream; + } + bool isOutputActive(IPU3CameraData *data) + { + return (data->activeStreamsMask & IPU3_STREAM_OUTPUT) ? + true : false; + } + void setOutputActive(IPU3CameraData *data) + { + data->activeStreamsMask |= IPU3_STREAM_OUTPUT; + } + bool isViewfinder(IPU3CameraData *data, Stream *stream) + { + return &data->vfStream_ == stream; + } + bool isViewfinderActive(IPU3CameraData *data) + { + return (data->activeStreamsMask & IPU3_STREAM_VF) ? + true : false; + } + void setViewfinderActive(IPU3CameraData *data) + { + data->activeStreamsMask |= IPU3_STREAM_VF; + } + bool isStreamActive(IPU3CameraData *data, Stream *stream) + { + if (isOutput(data, stream) && + isOutputActive(data)) + return true; + if (isViewfinder(data, stream) && + isViewfinderActive(data)) + return true; + + return false; + } + int registerCameras(); ImgUDevice imgu0_; @@ -210,7 +254,7 @@ PipelineHandlerIPU3::streamConfiguration(Camera *camera, { std::map configs; IPU3CameraData *data = cameraData(camera); - StreamConfiguration *config = &configs[&data->stream_]; + StreamConfiguration config = {}; /* * FIXME: Soraka: the maximum resolution reported by both sensors @@ -220,52 +264,93 @@ PipelineHandlerIPU3::streamConfiguration(Camera *camera, * * \todo Clarify ImgU alignement requirements. */ - config->width = 2560; - config->height = 1920; - config->pixelFormat = V4L2_PIX_FMT_NV12; - config->bufferCount = IPU3_BUFFER_COUNT; + config.width = 2560; + config.height = 1920; + config.pixelFormat = V4L2_PIX_FMT_NV12; + config.bufferCount = IPU3_BUFFER_COUNT; + + configs[&data->outStream_] = config; + LOG(IPU3, Debug) + << "Stream 'output' format set to " << config.width << "x" + << config.height << "-0x" << std::hex << std::setfill('0') + << std::setw(8) << config.pixelFormat; + configs[&data->vfStream_] = config; LOG(IPU3, Debug) - << "Stream format set to " << config->width << "x" - << config->height << "-0x" << std::hex << std::setfill('0') - << std::setw(8) << config->pixelFormat; + << "Stream 'viewfinder' format set to " << config.width << "x" + << config.height << "-0x" << std::hex << std::setfill('0') + << std::setw(8) << config.pixelFormat; return configs; } int PipelineHandlerIPU3::configureStreams(Camera *camera, - std::map &config) + std::map &config) { IPU3CameraData *data = cameraData(camera); - const StreamConfiguration &cfg = config[&data->stream_]; + StreamConfiguration CIO2Config = {}; CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; int ret; - LOG(IPU3, Info) - << "Requested image format " << cfg.width << "x" - << cfg.height << "-0x" << std::hex << std::setfill('0') - << std::setw(8) << cfg.pixelFormat << " on camera '" - << camera->name() << "'"; + /* Remove previously configured stream masks to store the new ones. */ + data->activeStreamsMask = 0; + for (auto const &streamConfig : config) { + Stream *stream = streamConfig.first; + const StreamConfiguration &cfg = streamConfig.second; - /* - * Verify that the requested size respects the IPU3 alignement - * requirements (the image width shall be a multiple of 8 pixels and - * its height a multiple of 4 pixels) and the camera maximum sizes. - * - * \todo: consider the BDS scaling factor requirements: - * "the downscaling factor must be an integer value multiple of 1/32" - */ - if (cfg.width % 8 || cfg.height % 4) { - LOG(IPU3, Error) << "Invalid stream size: bad alignment"; - return -EINVAL; - } + /* + * Verify that the requested size respects the IPU3 alignement + * requirements (the image width shall be a multiple of 8 + * pixels and its height a multiple of 4 pixels) and the camera + * maximum sizes. + * + * \todo: consider the BDS scaling factor requirements: "the + * downscaling factor must be an integer value multiple of + * 1/32" + */ + if (cfg.width % 8 || cfg.height % 4) { + LOG(IPU3, Error) + << "Invalid stream size: bad alignment"; + return -EINVAL; + } - if (cfg.width > cio2->maxSize_.width || - cfg.height > cio2->maxSize_.height) { - LOG(IPU3, Error) - << "Invalid stream size: larger than sensor resolution"; - return -EINVAL; + if (cfg.width > cio2->maxSize_.width || + cfg.height > cio2->maxSize_.height) { + LOG(IPU3, Error) + << "Invalid stream size: larger than sensor resolution"; + return -EINVAL; + } + + LOG(IPU3, Info) + << "Requested image format " << cfg.width << "x" + << cfg.height << "-0x" << std::hex << std::setw(8) + << cfg.pixelFormat << " on camera'" + << camera->name() << "'"; + + /* + * FIXME: As viewfinder should be operated even when + * applications do not intend to use it, we need to keep track + * of which streams have to be configured, to make meaningful + * decisions at configure and request queueing time. + * + * Walk here all the streams to configure and collect the + * active ones in a bitmaks. + */ + if (isOutput(data, stream)) + setOutputActive(data); + if (isViewfinder(data, stream)) + setViewfinderActive(data); + + /* + * Collect the maximum width and height: IPU3 can downscale + * only. + */ + if (cfg.width > CIO2Config.width) + CIO2Config.width = cfg.width; + if (cfg.height > CIO2Config.height) + CIO2Config.height = cfg.height; } /* @@ -281,26 +366,62 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, * adjusted format to be propagated to the ImgU output devices. */ V4L2DeviceFormat cio2Format = {}; - ret = cio2->configure(cfg, &cio2Format); + ret = cio2->configure(CIO2Config, &cio2Format); if (ret) return ret; - ret = imgu->configureInput(cfg, &cio2Format); + ret = imgu->configureInput(CIO2Config, &cio2Format); if (ret) return ret; /* Apply the format to the ImgU output, viewfinder and stat. */ - ret = imgu->configureOutput(&imgu->output_, cfg); - if (ret) - return ret; + for (auto const &streamConfig : config) { + Stream *stream = streamConfig.first; + const StreamConfiguration &cfg = streamConfig.second; - ret = imgu->configureOutput(&imgu->viewfinder_, cfg); - if (ret) - return ret; + if (isOutput(data, stream)) { + ret = imgu->configureOutput(&imgu->output_, cfg); + if (ret) + return ret; - ret = imgu->configureOutput(&imgu->stat_, cfg); - if (ret) - return ret; + ret = imgu->configureOutput(&imgu->stat_, cfg); + if (ret) + return ret; + + + if (isViewfinderActive(data)) + continue; + + /* + * Configure viewfinder using the output stream + * configuration if it is not part of the active + * streams list. + */ + ret = imgu->configureOutput(&imgu->viewfinder_, cfg); + if (ret) + return ret; + } else if (isViewfinder(data, stream)) { + ret = imgu->configureOutput(&imgu->viewfinder_, cfg); + if (ret) + return ret; + + if (isOutputActive(data)) + continue; + + /* + * Configure output using the viewfinder stream + * configuration if it is not part of the active + * streams list. + */ + ret = imgu->configureOutput(&imgu->output_, cfg); + if (ret) + return ret; + + ret = imgu->configureOutput(&imgu->stat_, cfg); + if (ret) + return ret; + } + } return 0; } @@ -404,7 +525,7 @@ int PipelineHandlerIPU3::queueRequest(Camera *camera, Request *request) { IPU3CameraData *data = cameraData(camera); V4L2Device *output = data->imgu_->output_.dev; - Stream *stream = &data->stream_; + Stream *stream = &data->outStream_; /* Queue a buffer to the ImgU output for capture. */ Buffer *buffer = request->findBuffer(stream); @@ -554,7 +675,10 @@ int PipelineHandlerIPU3::registerCameras() for (unsigned int id = 0; id < 4 && numCameras < 2; ++id) { std::unique_ptr data = utils::make_unique(this); - std::set streams{ &data->stream_ }; + std::set streams = { + &data->outStream_, + &data->vfStream_, + }; CIO2Device *cio2 = &data->cio2_; ret = cio2->init(cio2MediaDev_.get(), id); From patchwork Wed Apr 3 15:07:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 908 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 91A846110D for ; Wed, 3 Apr 2019 17:06:57 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id E760AFF804; Wed, 3 Apr 2019 15:06:56 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:30 +0200 Message-Id: <20190403150735.27580-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/8] libcamera: camera: allocateBuffers: Pass the stream set X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:06:58 -0000 Pipeline handlers might need to perform allocation of internal buffers, setup operations, or simple sanity check before going into the per-stream buffer allocation. As of now, PipelineHandler::allocateBuffers() is called once per each active stream, leaving no space for stream-independent configuration. Change this by providing to the pipeline handler the full set of active stream, and ask them to loop over them to perform per-streams allocations after eventual stream-independent operations. The same rational applies to PipelineHandler::freeBuffers() and eventual stream-independent clean up operations. While at it, change the return type of freeBuffers() to void as it was not checked by the Camera class method calling it. Reviewed-by: Niklas Söderlund Signed-off-by: Jacopo Mondi --- src/libcamera/camera.cpp | 15 +++++++-------- src/libcamera/include/pipeline_handler.h | 6 ++++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 15 +++++++++------ src/libcamera/pipeline/uvcvideo.cpp | 15 ++++++++++----- src/libcamera/pipeline/vimc.cpp | 15 ++++++++++----- src/libcamera/pipeline_handler.cpp | 8 ++++---- 6 files changed, 44 insertions(+), 30 deletions(-) diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 8ee9cc086616..99eb0e6876f0 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -467,13 +467,11 @@ int Camera::allocateBuffers() return -EINVAL; } - for (Stream *stream : activeStreams_) { - int ret = pipe_->allocateBuffers(this, stream); - if (ret) { - LOG(Camera, Error) << "Failed to allocate buffers"; - freeBuffers(); - return ret; - } + int ret = pipe_->allocateBuffers(this, activeStreams_); + if (ret) { + LOG(Camera, Error) << "Failed to allocate buffers"; + freeBuffers(); + return ret; } state_ = CameraPrepared; @@ -503,9 +501,10 @@ int Camera::freeBuffers() * by the V4L2 device that has allocated them. */ stream->bufferPool().destroyBuffers(); - pipe_->freeBuffers(this, stream); } + pipe_->freeBuffers(this, activeStreams_); + state_ = CameraConfigured; return 0; diff --git a/src/libcamera/include/pipeline_handler.h b/src/libcamera/include/pipeline_handler.h index acb376e07030..920b57609470 100644 --- a/src/libcamera/include/pipeline_handler.h +++ b/src/libcamera/include/pipeline_handler.h @@ -57,8 +57,10 @@ public: virtual int configureStreams(Camera *camera, std::map &config) = 0; - virtual int allocateBuffers(Camera *camera, Stream *stream) = 0; - virtual int freeBuffers(Camera *camera, Stream *stream) = 0; + virtual int allocateBuffers(Camera *camera, + const std::set &streams) = 0; + virtual void freeBuffers(Camera *camera, + const std::set &streams) = 0; virtual int start(Camera *camera) = 0; virtual void stop(Camera *camera); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index caf1051c58ab..a12e3f9a496d 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -145,8 +145,10 @@ public: int configureStreams(Camera *camera, std::map &config) override; - int allocateBuffers(Camera *camera, Stream *stream) override; - int freeBuffers(Camera *camera, Stream *stream) override; + int allocateBuffers(Camera *camera, + const std::set &streams) override; + void freeBuffers(Camera *camera, + const std::set &streams) override; int start(Camera *camera) override; void stop(Camera *camera) override; @@ -426,9 +428,11 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, return 0; } -int PipelineHandlerIPU3::allocateBuffers(Camera *camera, Stream *stream) +int PipelineHandlerIPU3::allocateBuffers(Camera *camera, + const std::set &streams) { IPU3CameraData *data = cameraData(camera); + Stream *stream = *streams.begin(); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; int ret; @@ -467,14 +471,13 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera, Stream *stream) return 0; } -int PipelineHandlerIPU3::freeBuffers(Camera *camera, Stream *stream) +void PipelineHandlerIPU3::freeBuffers(Camera *camera, + const std::set &streams) { IPU3CameraData *data = cameraData(camera); data->cio2_.freeBuffers(); data->imgu_->freeBuffers(); - - return 0; } int PipelineHandlerIPU3::start(Camera *camera) diff --git a/src/libcamera/pipeline/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo.cpp index cc3e0cd9afab..128f0c49dba3 100644 --- a/src/libcamera/pipeline/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo.cpp @@ -32,8 +32,10 @@ public: int configureStreams(Camera *camera, std::map &config) override; - int allocateBuffers(Camera *camera, Stream *stream) override; - int freeBuffers(Camera *camera, Stream *stream) override; + int allocateBuffers(Camera *camera, + const std::set &streams) override; + void freeBuffers(Camera *camera, + const std::set &streams) override; int start(Camera *camera) override; void stop(Camera *camera) override; @@ -129,9 +131,11 @@ int PipelineHandlerUVC::configureStreams(Camera *camera, return 0; } -int PipelineHandlerUVC::allocateBuffers(Camera *camera, Stream *stream) +int PipelineHandlerUVC::allocateBuffers(Camera *camera, + const std::set &streams) { UVCCameraData *data = cameraData(camera); + Stream *stream = *streams.begin(); const StreamConfiguration &cfg = stream->configuration(); LOG(UVC, Debug) << "Requesting " << cfg.bufferCount << " buffers"; @@ -139,10 +143,11 @@ int PipelineHandlerUVC::allocateBuffers(Camera *camera, Stream *stream) return data->video_->exportBuffers(&stream->bufferPool()); } -int PipelineHandlerUVC::freeBuffers(Camera *camera, Stream *stream) +void PipelineHandlerUVC::freeBuffers(Camera *camera, + const std::set &streams) { UVCCameraData *data = cameraData(camera); - return data->video_->releaseBuffers(); + data->video_->releaseBuffers(); } int PipelineHandlerUVC::start(Camera *camera) diff --git a/src/libcamera/pipeline/vimc.cpp b/src/libcamera/pipeline/vimc.cpp index 2e8c26fb7c0b..6735940799d8 100644 --- a/src/libcamera/pipeline/vimc.cpp +++ b/src/libcamera/pipeline/vimc.cpp @@ -32,8 +32,10 @@ public: int configureStreams(Camera *camera, std::map &config) override; - int allocateBuffers(Camera *camera, Stream *stream) override; - int freeBuffers(Camera *camera, Stream *stream) override; + int allocateBuffers(Camera *camera, + const std::set &streams) override; + void freeBuffers(Camera *camera, + const std::set &streams) override; int start(Camera *camera) override; void stop(Camera *camera) override; @@ -129,9 +131,11 @@ int PipelineHandlerVimc::configureStreams(Camera *camera, return 0; } -int PipelineHandlerVimc::allocateBuffers(Camera *camera, Stream *stream) +int PipelineHandlerVimc::allocateBuffers(Camera *camera, + const std::set &streams) { VimcCameraData *data = cameraData(camera); + Stream *stream = *streams.begin(); const StreamConfiguration &cfg = stream->configuration(); LOG(VIMC, Debug) << "Requesting " << cfg.bufferCount << " buffers"; @@ -139,10 +143,11 @@ int PipelineHandlerVimc::allocateBuffers(Camera *camera, Stream *stream) return data->video_->exportBuffers(&stream->bufferPool()); } -int PipelineHandlerVimc::freeBuffers(Camera *camera, Stream *stream) +void PipelineHandlerVimc::freeBuffers(Camera *camera, + const std::set &streams) { VimcCameraData *data = cameraData(camera); - return data->video_->releaseBuffers(); + data->video_->releaseBuffers(); } int PipelineHandlerVimc::start(Camera *camera) diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 1a858f2638ce..9a8a4fde57e6 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -193,10 +193,10 @@ PipelineHandler::~PipelineHandler() * \fn PipelineHandler::allocateBuffers() * \brief Allocate buffers for a stream * \param[in] camera The camera the \a stream belongs to - * \param[in] stream The stream to allocate buffers for + * \param[in] streams The group of streams to allocate buffers for * * This method allocates buffers internally in the pipeline handler and - * associates them with the stream's buffer pool. + * associates them with each stream's buffer pool. * * The intended caller of this method is the Camera class. * @@ -207,9 +207,9 @@ PipelineHandler::~PipelineHandler() * \fn PipelineHandler::freeBuffers() * \brief Free all buffers associated with a stream * \param[in] camera The camera the \a stream belongs to - * \param[in] stream The stream to free buffers from + * \param[in] streams The group of streams to free buffers from * - * After a capture session has been stopped all buffers associated with the + * After a capture session has been stopped all buffers associated with each * stream shall be freed. * * The intended caller of this method is the Camera class. From patchwork Wed Apr 3 15:07:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 909 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 29B63610C5 for ; Wed, 3 Apr 2019 17:06:58 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id B87BDFF80B; Wed, 3 Apr 2019 15:06:57 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:31 +0200 Message-Id: <20190403150735.27580-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/8] libcamera: ipu3: Add multiple stream memory management X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:06:58 -0000 Perform allocation and setup of memory sharing betweent the CIO2 output and the ImgU input and allocate memory for each active stream. --- src/libcamera/pipeline/ipu3/ipu3.cpp | 78 ++++++++++++++++++++++------ 1 file changed, 62 insertions(+), 16 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a12e3f9a496d..f7e75fac1dfe 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -91,6 +91,7 @@ public: BufferPool vfPool_; BufferPool statPool_; + BufferPool outPool_; }; class CIO2Device @@ -428,13 +429,21 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, return 0; } +/** + * \todo Clarify if 'viewfinder' and 'stat' nodes have to be set up and + * started even if not in use. As of now, if not properly configured and + * enabled, the ImgU processing pipeline stalls. + * + * In order to be able to start the 'viewfinder' and 'stat' nodes, we need + * memory to be reserved. + */ int PipelineHandlerIPU3::allocateBuffers(Camera *camera, const std::set &streams) { IPU3CameraData *data = cameraData(camera); - Stream *stream = *streams.begin(); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; + unsigned int bufferCount; int ret; /* Share buffers between CIO2 output and ImgU input. */ @@ -446,28 +455,64 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera, if (ret) return ret; - /* Export ImgU output buffers to the stream's pool. */ - ret = imgu->exportBuffers(&imgu->output_, &stream->bufferPool()); - if (ret) - return ret; - /* - * Reserve memory in viewfinder and stat output devices. Use the - * same number of buffers as the ones requested for the output - * stream. + * Use for internal pools the same buffer number as the input + * one: it should match the pipeline lenght. */ - unsigned int bufferCount = stream->bufferPool().count(); - - imgu->viewfinder_.pool->createBuffers(bufferCount); - ret = imgu->exportBuffers(&imgu->viewfinder_, imgu->viewfinder_.pool); - if (ret) - return ret; - + bufferCount = pool->count(); imgu->stat_.pool->createBuffers(bufferCount); ret = imgu->exportBuffers(&imgu->stat_, imgu->stat_.pool); if (ret) return ret; + for (Stream *stream : streams) { + if (isOutput(data, stream)) { + /* Export output buffers to the stream's pool. */ + ret = imgu->exportBuffers(&imgu->output_, + &stream->bufferPool()); + if (ret) + return ret; + + if (isViewfinderActive(data)) + continue; + + /* + * Reserve memory in viewfinder device if it is not + * part of the active streams list. Use the same + * number of buffers as the ones requested for the + * output stream. + */ + bufferCount = stream->bufferPool().count(); + imgu->viewfinder_.pool->createBuffers(bufferCount); + ret = imgu->exportBuffers(&imgu->viewfinder_, + imgu->viewfinder_.pool); + if (ret) + return ret; + } else if (isViewfinder(data, stream)) { + /* Export viewfinder buffers to the stream's pool. */ + ret = imgu->exportBuffers(&imgu->viewfinder_, + &stream->bufferPool()); + if (ret) + return ret; + + if (isOutputActive(data)) + continue; + + /* + * Reserve memory in output device if it is not part + * of the active streams list. Use the same number + * of buffers as the ones requested for the viewfinder + * stream. + */ + bufferCount = stream->bufferPool().count(); + imgu->output_.pool->createBuffers(bufferCount); + ret = imgu->exportBuffers(&imgu->output_, + imgu->output_.pool); + if (ret) + return ret; + } + } + return 0; } @@ -819,6 +864,7 @@ int ImgUDevice::init(MediaDevice *media, unsigned int index) output_.pad = PAD_OUTPUT; output_.name = "output"; + output_.pool = &outPool_; viewfinder_.dev = V4L2Device::fromEntityName(media, name_ + " viewfinder"); From patchwork Wed Apr 3 15:07:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 910 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BEE3D6110D for ; Wed, 3 Apr 2019 17:06:58 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 54BE8FF806; Wed, 3 Apr 2019 15:06:58 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:32 +0200 Message-Id: <20190403150735.27580-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/8] libcamera: request: Add streams() method X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:06:59 -0000 Add a method to retrieve the list of streams contained in a requests. This is useful for pipeline handler willing to cycle on all the streams a request contains at queueRequest() time. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- include/libcamera/request.h | 3 +++ src/libcamera/request.cpp | 15 +++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/libcamera/request.h b/include/libcamera/request.h index 0dbd425115e8..5ac4d20d1d9f 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -7,6 +7,7 @@ #ifndef __LIBCAMERA_REQUEST_H__ #define __LIBCAMERA_REQUEST_H__ +#include #include #include @@ -35,6 +36,8 @@ public: int setBuffers(const std::map &streamMap); Buffer *findBuffer(Stream *stream) const; + const std::list streams() const; + Status status() const { return status_; } private: diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index e0e77e972411..3a7841fb2bb3 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -93,6 +93,21 @@ Buffer *Request::findBuffer(Stream *stream) const return it->second; } +/** + * \brief Retrieve the set of streams contained in the request + * + * \return The set of streams contained in the request + */ +const std::list Request::streams() const +{ + std::list streams = {}; + + for (auto const &it : bufferMap_) + streams.push_front(it.first); + + return streams; +} + /** * \fn Request::status() * \brief Retrieve the request completion status From patchwork Wed Apr 3 15:07:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 911 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6484A610BF for ; Wed, 3 Apr 2019 17:06:59 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id EC96CFF806; Wed, 3 Apr 2019 15:06:58 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:33 +0200 Message-Id: <20190403150735.27580-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/8] libcamera: ipu3: Queue request for multiple streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:07:00 -0000 Add support for queue request on the main and secondary outputs of the ImgU. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f7e75fac1dfe..8c67cf985d1e 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -572,20 +572,22 @@ void PipelineHandlerIPU3::stop(Camera *camera) int PipelineHandlerIPU3::queueRequest(Camera *camera, Request *request) { IPU3CameraData *data = cameraData(camera); - V4L2Device *output = data->imgu_->output_.dev; - Stream *stream = &data->outStream_; - /* Queue a buffer to the ImgU output for capture. */ - Buffer *buffer = request->findBuffer(stream); - if (!buffer) { - LOG(IPU3, Error) - << "Attempt to queue request with invalid stream"; - return -ENOENT; - } + for (Stream *stream : request->streams()) { + Buffer *buffer = request->findBuffer(stream); + if (!buffer) { + LOG(IPU3, Error) << "Attempt to queue invalid request"; + return -ENOENT; + } - int ret = output->queueBuffer(buffer); - if (ret < 0) - return ret; + V4L2Device *videoDevice = isOutput(data, stream) + ? data->imgu_->output_.dev + : data->imgu_->viewfinder_.dev; + + int ret = videoDevice->queueBuffer(buffer); + if (ret) + return ret; + } PipelineHandler::queueRequest(camera, request); From patchwork Wed Apr 3 15:07:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 912 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08E4E610C5 for ; Wed, 3 Apr 2019 17:07:00 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 903A5FF80B; Wed, 3 Apr 2019 15:06:59 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:34 +0200 Message-Id: <20190403150735.27580-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 7/8] libcamera: pipeline: Add method to retrieve Request from Buffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:07:00 -0000 Add a method to CameraData base class to retrieve a pointer to the Request that contains a given buffer. Intended users are buffer completion slots that needs to associate a Request to a just completed Buffer. In preparation to support multiple requests from different streams, update all the pipeline handler implementations to use this method instead of using the last queued request. Signed-off-by: Jacopo Mondi --- include/libcamera/request.h | 2 ++ src/libcamera/include/pipeline_handler.h | 3 +++ src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- src/libcamera/pipeline/uvcvideo.cpp | 3 ++- src/libcamera/pipeline/vimc.cpp | 3 ++- src/libcamera/pipeline_handler.cpp | 29 ++++++++++++++++++++++++ src/libcamera/request.cpp | 7 ++++++ 7 files changed, 47 insertions(+), 3 deletions(-) diff --git a/include/libcamera/request.h b/include/libcamera/request.h index 5ac4d20d1d9f..8f5892fd3111 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -38,6 +38,8 @@ public: const std::list streams() const; + const std::unordered_set &pending() const { return pending_; } + Status status() const { return status_; } private: diff --git a/src/libcamera/include/pipeline_handler.h b/src/libcamera/include/pipeline_handler.h index 920b57609470..6cdadcbdc3ea 100644 --- a/src/libcamera/include/pipeline_handler.h +++ b/src/libcamera/include/pipeline_handler.h @@ -39,6 +39,9 @@ public: PipelineHandler *pipe_; std::list queuedRequests_; +protected: + Request *requestFromBuffer(Buffer *buffer); + private: CameraData(const CameraData &) = delete; CameraData &operator=(const CameraData &) = delete; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 8c67cf985d1e..17e3e8677e28 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -801,7 +801,8 @@ void PipelineHandlerIPU3::IPU3CameraData::imguInputBufferReady(Buffer *buffer) */ void PipelineHandlerIPU3::IPU3CameraData::imguOutputBufferReady(Buffer *buffer) { - Request *request = queuedRequests_.front(); + Request *request = requestFromBuffer(buffer); + ASSERT(request); pipe_->completeBuffer(camera_, request, buffer); pipe_->completeRequest(camera_, request); diff --git a/src/libcamera/pipeline/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo.cpp index 128f0c49dba3..d571b8b4ea83 100644 --- a/src/libcamera/pipeline/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo.cpp @@ -226,7 +226,8 @@ bool PipelineHandlerUVC::match(DeviceEnumerator *enumerator) void PipelineHandlerUVC::UVCCameraData::bufferReady(Buffer *buffer) { - Request *request = queuedRequests_.front(); + Request *request = requestFromBuffer(buffer); + ASSERT(request); pipe_->completeBuffer(camera_, request, buffer); pipe_->completeRequest(camera_, request); diff --git a/src/libcamera/pipeline/vimc.cpp b/src/libcamera/pipeline/vimc.cpp index 6735940799d8..e83416effad8 100644 --- a/src/libcamera/pipeline/vimc.cpp +++ b/src/libcamera/pipeline/vimc.cpp @@ -223,7 +223,8 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) void PipelineHandlerVimc::VimcCameraData::bufferReady(Buffer *buffer) { - Request *request = queuedRequests_.front(); + Request *request = requestFromBuffer(buffer); + ASSERT(request); pipe_->completeBuffer(camera_, request, buffer); pipe_->completeRequest(camera_, request); diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 9a8a4fde57e6..830ff354ed3e 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -86,6 +86,35 @@ LOG_DEFINE_CATEGORY(Pipeline) * PipelineHandler::completeRequest() */ +/** + * \brief Retrieve the pending request that contains \a buffer + * \param[in] buffer The buffer contained in the returned request + * + * Return the request that contains \a buffer, or nullptr if no such request + * is found. The intended callers of this method are buffer completion slots + * implemented in CameraData sub-classes which needs to associated a request + * to the just completed buffer. It is up to the caller of this function to + * deal with the case the buffer does not belong to any previously queued + * request or the request has already completed, possibly because of a + * duplicated buffer completion notification. This is generally considered + * a fatal error, and callers are expected to assert the validity of the + * returned request. + * + * \return A pointer to the pending Request that contains the Buffer \a buffer, + * or nullptr if no such request is found + */ +Request *CameraData::requestFromBuffer(Buffer *buffer) +{ + for (Request *req : queuedRequests_) { + for (Buffer *b : req->pending()) { + if (b == buffer) + return req; + } + } + + return nullptr; +} + /** * \class PipelineHandler * \brief Create and manage cameras based on a set of media devices diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 3a7841fb2bb3..c555752b2c0b 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -108,6 +108,13 @@ const std::list Request::streams() const return streams; } +/** + * \fn Request::pending() + * \brief Retrieve the list of not-yet-completed buffers + * + * \return The set of pending buffers + */ + /** * \fn Request::status() * \brief Retrieve the request completion status From patchwork Wed Apr 3 15:07:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 913 Return-Path: Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A4D75610BF for ; Wed, 3 Apr 2019 17:07:00 +0200 (CEST) X-Originating-IP: 2.224.242.101 Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 30C33FF817; Wed, 3 Apr 2019 15:07:00 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Apr 2019 17:07:35 +0200 Message-Id: <20190403150735.27580-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190403150735.27580-1-jacopo@jmondi.org> References: <20190403150735.27580-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/8] libcamera: ipu3: Connect viewfinder's BufferReady signal X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:07:00 -0000 Connect the viewfinder buffer ready signal to the IPU3CameraData slot that complets the buffer first, and if not waiting for other buffers completes the request as well. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 17e3e8677e28..706e4f647ed7 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -167,7 +167,7 @@ private: { } - void imguOutputBufferReady(Buffer *buffer); + void imguCaptureBufferReady(Buffer *buffer); void imguInputBufferReady(Buffer *buffer); void cio2BufferReady(Buffer *buffer); @@ -755,7 +755,9 @@ int PipelineHandlerIPU3::registerCameras() data->imgu_->input_->bufferReady.connect(data.get(), &IPU3CameraData::imguInputBufferReady); data->imgu_->output_.dev->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); + &IPU3CameraData::imguCaptureBufferReady); + data->imgu_->viewfinder_.dev->bufferReady.connect(data.get(), + &IPU3CameraData::imguCaptureBufferReady); /* Create and register the Camera instance. */ std::string cameraName = cio2->sensor_->entityName() + " " @@ -799,13 +801,13 @@ void PipelineHandlerIPU3::IPU3CameraData::imguInputBufferReady(Buffer *buffer) * * Buffers completed from the ImgU output are directed to the application. */ -void PipelineHandlerIPU3::IPU3CameraData::imguOutputBufferReady(Buffer *buffer) +void PipelineHandlerIPU3::IPU3CameraData::imguCaptureBufferReady(Buffer *buffer) { Request *request = requestFromBuffer(buffer); ASSERT(request); - pipe_->completeBuffer(camera_, request, buffer); - pipe_->completeRequest(camera_, request); + if (pipe_->completeBuffer(camera_, request, buffer)) + pipe_->completeRequest(camera_, request); } /**