From patchwork Wed Mar 20 16:30:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 746 Return-Path: Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C50A3600FC for ; Wed, 20 Mar 2019 17:30:27 +0100 (CET) Received: from uno.lan (2-224-242-101.ip172.fastwebnet.it [2.224.242.101]) (Authenticated sender: jacopo@jmondi.org) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 5188C200010; Wed, 20 Mar 2019 16:30:27 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Mar 2019 17:30:24 +0100 Message-Id: <20190320163055.22056-1-jacopo@jmondi.org> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 00/31] libcamera: ipu3: Add ImgU support + multiple streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2019 16:30:28 -0000 Hello, I know, 31 patches, it's bad, but please don't run away. Not yet at least. This series combines a few things, I decided to keep it together as one part is based on the other, but they could be actually considered separate. The series includes: - Support for IPU3 ImgU unit (patches 01->15) I have sent v1 some time ago, v2 not too long ago and an updated version called v3-part which included and updated version of the first 4 patches only. I have updated the v3-part patches with: - Remove all formats not supported by the IPU3 (only keep 10-bit RAW Bayer) - Standardized printout of formats to remove space wherever possible - Add Kieran's tags Patches from 04->15 are similar to the ones in v2 with small mostly cosmetic updates. Please review these ones if you've not tackled v2 already. - Support for multiple stream capture in IPU3 pipeline handler (16->26) This part contains a few small changes I consider for inclusion for the Request class and a few I would like to receive feedback for the Pipeline Handler base class to add pre/post memory allocation hooks (marked as RFC). All other patches extends the IPU3 pipeline handler to work with both output and viewfinder, with the quirk that I need to operate the viewfinder and the stat node even when not required as otherwise capture operations stall. - Hacks on the cam application to support capture of multiple streams (26->31) Mostly hacks to enable my testing, with the very bad ones marked as HACK and the remaining ones I hope might be useful to Niklas to extend format parsing to support multiple streams. Tested on Soraka capturing output, viewfinder and output+viewfinder at the same time in 2560x1920, 1920x1080 and 640x480 resolutions. A few points still bother me: - viewfinder and stat nodes needs to be operated anyhow, otherwise ImgU does work and stall. - It is not yet clear to me how to calculate the sizes in the capture pipeline based on the requested output resolutions. Upstream pointed us to an XML file with all sizes hardcoded, which is an unacceptable solution as sizes have to be calculated dynamically based on requested stream configurations. - When operating on one of the two ImgU pipes, if the other one is still linked from a previous capture operation, the whole ImgU unit stall. This is maybe expected, but I have not found yet a reasonable way to handle this. Please see 15/31 (marked as RFC for this reason) where I try to explain the issue in a lengthy comment. I'll do more testing on the above points and get in contact with IPU3 upstream driver developers soon to have clarifications, in the meantime, feedbacks are welcome. Thanks j Jacopo Mondi (31): libcamera: formats: Add toString() methods libcamera: formats: Define FormatEnum type libcamera: ipu3: Make sure sensor provides a compatible format libcamera: ipu3: Set stream configuration from sensor libcamera: ipu3: Initialize and configure CIO2 libcamera: ipu3: Initialize and configure ImgUs libcamera: ipu3: Propagate image format libcamera: ipu3: Implement camera start/stop libcamera: ipu3: Implement buffer allocation libcamera: ipu3: Implement buffer release libcamera: ipu3: Queue requests to the pipeline libcamera: ipu3: Connect CIO2 and ImgU bufferReady signals libcamera: ipu3: Use NV12 as default image format HACK: Soraka: Limit resolution to 2560x1920 RFC: libcamera: ipu3: Enable ImgU media links libcamera: request: Store the streams in the request libcamera: request: Add camera() getter method libcamera: ipu3: Create camera with 2 streams libcamera: ipu3: Configure 'output' and 'viewfinder' libcamera: ipu3: Add multiple stream memory management RFC: libcamera: pipeline_handlers: Add preAllocateBuffers RFC: libcamera: pipeline_handlers: Add postFreeBuffers libcamera: ipu3: Enable media links conditionally libcamera: ipu3: Queue request for multiple streams libcamera: ipu3: Connect viewfinder's BufferReady signal HACK: cam: Allow camera with multiple streams src: cam: Create requests with multiple streams src: cam: Add output name to frame writer src: cam: Handle multiple streams in request complete HACK: src: cam: Play with streams combinations HACK: cam: Add options to select the stream to use include/libcamera/request.h | 3 + src/cam/buffer_writer.cpp | 4 +- src/cam/buffer_writer.h | 2 +- src/cam/main.cpp | 132 ++- src/libcamera/camera.cpp | 12 +- src/libcamera/formats.cpp | 27 + src/libcamera/include/formats.h | 22 + src/libcamera/include/pipeline_handler.h | 9 + src/libcamera/include/v4l2_device.h | 2 + src/libcamera/include/v4l2_subdevice.h | 6 +- src/libcamera/meson.build | 1 + src/libcamera/pipeline/ipu3/ipu3.cpp | 1383 +++++++++++++++++++--- src/libcamera/pipeline_handler.cpp | 40 +- src/libcamera/request.cpp | 23 + src/libcamera/v4l2_device.cpp | 18 + src/libcamera/v4l2_subdevice.cpp | 23 +- 16 files changed, 1478 insertions(+), 229 deletions(-) create mode 100644 src/libcamera/formats.cpp create mode 100644 src/libcamera/include/formats.h --- 2.21.0