From patchwork Wed Aug 5 15:14:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9228 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9EEF9BD86F for ; Wed, 5 Aug 2020 15:15:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 68BB4605C0; Wed, 5 Aug 2020 17:15:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Y6rit37R"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 497F360392 for ; Wed, 5 Aug 2020 17:15:11 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C312D304; Wed, 5 Aug 2020 17:15:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640511; bh=DxAIImIc1YcTwfXBvA42JnIiRQoaoMCbU5KR6C5iMW4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y6rit37RMCymzw6+bQB+MLvWtSlAXIOhhg8MN89Jvn5FQKe0J/EOFcndlJBi7cgv+ eYtCEQf1/7nllPTU095TX9M9Opzu5ESjePIiIp8whm3WmSiyVs3P1IvlNCr3lzJSCx ASGizU1jwVu9LMRo1qNP+DiBhsVSqeKgmB8gd1p4= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:14:55 +0100 Message-Id: <20200805151507.227503-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 01/13] libcamera: buffer: Correct FrameBuffer references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the remaining (capitalised) Buffer references to the new FrameBuffer object that they refer to. Lower-case 'buffer' terms are kept, as a generic reference to a buffer rather than the specific type 'FrameBuffer'. Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- src/libcamera/buffer.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 1a1d4bac7aed..8278f8a92af4 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -174,11 +174,11 @@ FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie) * The intended callers of this method are buffer completion handlers that * need to associate a buffer to the request it belongs to. * - * A Buffer is associated to a request by Request::addBuffer() and the + * A FrameBuffer is associated to a request by Request::addBuffer() and the * association is valid until the buffer completes. The returned request * pointer is valid only during that interval. * - * \return The Request the Buffer belongs to, or nullptr if the buffer is + * \return The Request the FrameBuffer belongs to, or nullptr if the buffer is * not associated with a request */ @@ -224,7 +224,7 @@ FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie) /** * \brief Copy the contents from another buffer - * \param[in] src Buffer to copy + * \param[in] src FrameBuffer to copy * * Copy the buffer contents and metadata from \a src to this buffer. The * destination FrameBuffer shall have the same number of planes as the source From patchwork Wed Aug 5 15:14:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9229 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 07A49BD87B for ; Wed, 5 Aug 2020 15:15:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BD9F460745; Wed, 5 Aug 2020 17:15:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gcVmEgdo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B5E5A60392 for ; Wed, 5 Aug 2020 17:15:11 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 389ACA5E; Wed, 5 Aug 2020 17:15:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640511; bh=m5BBFcRsKZ1wbFyRcIpoeZeom2pKehtEEUeN9KVELjw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gcVmEgdoWnOe8KaYIi+kzElGh+glNiWxwfJMUaKk6QVxObQ7aptgDGrf4k90JbOK5 reyCjEiocGhBN232e4bjnRX8R/mOqzZ16uW1v0TuTcBUFfHe4LRnSDNEyee70b08zF mDdHBgzXr3uryGPfGtvdVizQvA6ELsFMsc4PvLWc= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:14:56 +0100 Message-Id: <20200805151507.227503-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/13] libcamera: formats: add numPlanes helper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Determine the number of planes used by a format by counting the number of PixelFormatPlaneInfo entries with a valid entry. Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- v3: - Rename $SUBJECT :D v2: - Rename to numPlanes - Simplify fucntion doc. include/libcamera/internal/formats.h | 2 ++ src/libcamera/formats.cpp | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h index b8e474916378..51a8a6b8b0ae 100644 --- a/include/libcamera/internal/formats.h +++ b/include/libcamera/internal/formats.h @@ -46,6 +46,8 @@ public: unsigned int frameSize(const Size &size, const std::array &strides) const; + unsigned int numPlanes() const; + /* \todo Add support for non-contiguous memory planes */ const char *name; PixelFormat format; diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp index a51e96810aed..ebaae9be3d75 100644 --- a/src/libcamera/formats.cpp +++ b/src/libcamera/formats.cpp @@ -821,4 +821,22 @@ PixelFormatInfo::frameSize(const Size &size, return sum; } +/** + * \brief Retrieve the number of planes represented by the format + * \return The number of planes used by the format + */ +unsigned int PixelFormatInfo::numPlanes() const +{ + unsigned int count = 0; + + for (const PixelFormatPlaneInfo &p : planes) { + if (p.bytesPerGroup == 0) + break; + + count++; + } + + return count; +} + } /* namespace libcamera */ From patchwork Wed Aug 5 15:14:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9230 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 458B2BD86F for ; Wed, 5 Aug 2020 15:15:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 140AF605B8; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QpVRn4Jz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 12C1660599 for ; Wed, 5 Aug 2020 17:15:12 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A46D8B90; Wed, 5 Aug 2020 17:15:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640511; bh=ybsy37bbS0FJ5GnFEaly8S/4eQECIx+M/OrEaRVRFmo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QpVRn4JzZMtaFqzPeitIzvMIGeSpQ7nvDmpY8LMd5d3sgCVnqnkMzb/jmqcdR9wVp qSive8DnAwAqrjy01YH8y3i2TAUP02k0z2gHUXuiHIpiKFWDfwyyXd6Vz+mlk/nPPV IK8g4AYbIlOKMJMw958ptHkFjFAL/tASSH3IiFEY= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:14:57 +0100 Message-Id: <20200805151507.227503-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/13] libcamera: buffer: Create a MappedBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a MappedFrameBuffer helper class which will map all of the Planes within a FrameBuffer and provide CPU addressable pointers for those planes. The MappedFrameBuffer implements the interface of the MappedBuffer allowing other buffer types to be constructed of the same form, with a common interface and cleanup. This allows MappedBuffer instances to be created from Camera3Buffer types. Mappings are removed upon destruction. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- v3 - Documentation fixups v4: - Use reference when unmapping, rather than a copy of the planes. - Move MappedPlane to MappedBuffer::Plane - Remove valid_ - Make constructor protected - Rename move constructor parameters from rhs to other - Rework move constructor documentation. --- include/libcamera/internal/buffer.h | 47 +++++++++ src/libcamera/buffer.cpp | 152 +++++++++++++++++++++++++++- 2 files changed, 198 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/buffer.h diff --git a/include/libcamera/internal/buffer.h b/include/libcamera/internal/buffer.h new file mode 100644 index 000000000000..b7b0173f93f5 --- /dev/null +++ b/include/libcamera/internal/buffer.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * buffer.h - Internal buffer handling + */ +#ifndef __LIBCAMERA_INTERNAL_BUFFER_H__ +#define __LIBCAMERA_INTERNAL_BUFFER_H__ + +#include +#include + +#include +#include + +namespace libcamera { + +class MappedBuffer +{ +public: + using Plane = Span; + + ~MappedBuffer(); + + MappedBuffer(MappedBuffer &&other); + MappedBuffer &operator=(MappedBuffer &&other); + + bool isValid() const { return error_ == 0; } + int error() const { return error_; } + const std::vector &maps() const { return maps_; } + +protected: + MappedBuffer(); + + int error_; + std::vector maps_; +}; + +class MappedFrameBuffer : public MappedBuffer +{ +public: + MappedFrameBuffer(const FrameBuffer *buffer, int flags); +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_BUFFER_H__ */ diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 8278f8a92af4..8666f1992a77 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -6,6 +6,7 @@ */ #include +#include "libcamera/internal/buffer.h" #include #include @@ -15,7 +16,8 @@ #include "libcamera/internal/log.h" /** - * \file buffer.h + * \file libcamera/buffer.h + * \file libcamera/internal/buffer.h * \brief Buffer handling */ @@ -290,4 +292,152 @@ int FrameBuffer::copyFrom(const FrameBuffer *src) return 0; } +/** + * \class MappedBuffer + * \brief Provide an interface to support managing memory mapped buffers + * + * The MappedBuffer interface provides access to a set of MappedPlanes which + * are available for access by the CPU. + * + * This class is not meant to be constructed directly, but instead derived + * classes should be used to implement the correct mapping of a source buffer. + * + * This allows treating CPU accessible memory through a generic interface + * regardless of whether it originates from a libcamera FrameBuffer or other + * source. + */ + +/** + * \typedef MappedBuffer::Plane + * \brief A mapped region of memory accessible to the CPU + * + * The MappedBuffer::Plane uses the Span interface to describe the mapped memory + * region. + */ + +/** + * \brief Construct an empty MappedBuffer + * + * Base class initialisation for MappedBuffer. + */ +MappedBuffer::MappedBuffer() + : error_(0) +{ +} + +/** + * \brief Move constructor, construct the MappedBuffer with the contents of \a + * other using move semantics + * \param[in] other The other MappedBuffer + * + * Moving a MappedBuffer moves the mappings contained in the \a other to the new + * MappedBuffer and invalidates the \a other. + * + * No mappings are unmapped or destroyed in this process. + */ +MappedBuffer::MappedBuffer(MappedBuffer &&other) +{ + *this = std::move(other); +} + +/** + * \brief Move assignment operator, replace the mappings with those of \a other + * mappings + * \param[in] other The other MappedBuffer + * + * Moving a MappedBuffer moves the mappings contained in the \a other to the new + * MappedBuffer and invalidates the \a other. + * + * No mappings are unmapped or destroyed in this process. + */ +MappedBuffer &MappedBuffer::operator=(MappedBuffer &&other) +{ + error_ = other.error_; + maps_ = std::move(other.maps_); + other.error_ = -ENOENT; + + return *this; +} + +MappedBuffer::~MappedBuffer() +{ + for (Plane &map : maps_) + munmap(map.data(), map.size()); +} + +/** + * \fn MappedBuffer::isValid() + * \brief Check if the MappedBuffer instance is valid + * \return True if the MappedBuffer has valid mappings, false otherwise + */ + +/** + * \fn MappedBuffer::error() + * \brief Retrieve the map error status + * + * This function retrieves the error status from the MappedBuffer. + * The error status is a negative number as defined by errno.h. If + * no error occurred, this function returns 0. + * + * \return 0 on success or a negative error code otherwise + */ + +/** + * \fn MappedBuffer::maps() + * \brief Retrieve the mapped planes + * + * This function retrieves the successfully mapped planes stored as a vector + * of Span to provide access to the mapped memory. + * + * \return A vector of the mapped planes + */ + +/** + * \var MappedBuffer::error_ + * \brief Stores the error value if present + * + * MappedBuffer derived classes shall set this to a negative value as defined + * by errno.h if an error occured during the mapping process. + */ + +/** + * \var MappedBuffer::maps_ + * \brief Stores the internal + * + * MappedBuffer derived classes shall store the mappings they create in this + * vector which is parsed during destruct to unmap any memory mappings which + * completed successfully. + */ + +/** + * \class MappedFrameBuffer + * \brief Map a FrameBuffer using the MappedBuffer interface + */ + +/** + * \brief Map all planes of a FrameBuffer + * \param[in] buffer FrameBuffer to be mapped + * \param[in] flags Protection flags to apply to map + * + * Construct an object to map a frame buffer for CPU access. + * The flags are passed directly to mmap and should be either PROT_READ, + * PROT_WRITE, or a bitwise-or combination of both. + */ +MappedFrameBuffer::MappedFrameBuffer(const FrameBuffer *buffer, int flags) +{ + maps_.reserve(buffer->planes().size()); + + for (const FrameBuffer::Plane &plane : buffer->planes()) { + void *address = mmap(nullptr, plane.length, flags, + MAP_SHARED, plane.fd.fd(), 0); + if (address == MAP_FAILED) { + error_ = -errno; + LOG(Buffer, Error) << "Failed to mmap plane"; + break; + } + + maps_.emplace_back(static_cast(address), plane.length); + } +} + } /* namespace libcamera */ From patchwork Wed Aug 5 15:14:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9231 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93A25BD87B for ; Wed, 5 Aug 2020 15:15:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 609C0607E7; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="q1cFixOz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8192C60599 for ; Wed, 5 Aug 2020 17:15:12 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 03952D2F; Wed, 5 Aug 2020 17:15:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640512; bh=gOM+OtUXdmlLUx5AKttWr5qtbwqqt0qdbGiYKpylbXs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q1cFixOzfdgvrfVe0WDHHni7Gj/MhFoElpf+XlVVWxvfJZ36WgrsrYTn/Xau77A7R OtgySu0uiyYkdgUNyeMIHA86kjyVqCoJakiaBguMehBkzibtSgjNBwDvZvtcfPIPLE /mRh+jF5H1Dyr1vfPpj2CJbaj2iFsF6qIBIYQdFU= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:14:58 +0100 Message-Id: <20200805151507.227503-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/13] test: mapped-buffers: Provide MappedBuffer test X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide initial testing framework for the MappedBuffer component. Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- test/mapped-buffer.cpp | 114 +++++++++++++++++++++++++++++++++++++++++ test/meson.build | 1 + 2 files changed, 115 insertions(+) create mode 100644 test/mapped-buffer.cpp diff --git a/test/mapped-buffer.cpp b/test/mapped-buffer.cpp new file mode 100644 index 000000000000..b1c13ad03e62 --- /dev/null +++ b/test/mapped-buffer.cpp @@ -0,0 +1,114 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * libcamera internal MappedBuffer tests + */ + +#include + +#include "libcamera/internal/buffer.h" + +#include "camera_test.h" +#include "test.h" + +using namespace std; + +namespace { + +class MappedBufferTest : public CameraTest, public Test +{ +public: + MappedBufferTest() + : CameraTest("VIMC Sensor B") + { + } + +protected: + int init() override + { + if (status_ != TestPass) + return status_; + + config_ = camera_->generateConfiguration({ StreamRole::VideoRecording }); + if (!config_ || config_->size() != 1) { + cout << "Failed to generate default configuration" << endl; + return TestFail; + } + + allocator_ = new FrameBufferAllocator(camera_); + + StreamConfiguration &cfg = config_->at(0); + + if (camera_->acquire()) { + cout << "Failed to acquire the camera" << endl; + return TestFail; + } + + if (camera_->configure(config_.get())) { + cout << "Failed to set default configuration" << endl; + return TestFail; + } + + stream_ = cfg.stream(); + + int ret = allocator_->allocate(stream_); + if (ret < 0) + return TestFail; + + return TestPass; + } + + void cleanup() override + { + delete allocator_; + } + + int run() override + { + const std::unique_ptr &buffer = allocator_->buffers(stream_).front(); + std::vector maps; + + MappedFrameBuffer map(buffer.get(), PROT_READ); + if (!map.isValid()) { + cout << "Failed to successfully map buffer" << endl; + return TestFail; + } + + /* Make sure we can move it. */ + maps.emplace_back(std::move(map)); + + /* But copying is prevented, it would cause double-unmap. */ + // MappedFrameBuffer map_copy = map; + + /* Local map should be invalid (after move). */ + if (map.isValid()) { + cout << "Post-move map should not be valid" << endl; + return TestFail; + } + + /* Test for multiple successful maps on the same buffer. */ + MappedFrameBuffer write_map(buffer.get(), PROT_WRITE); + if (!write_map.isValid()) { + cout << "Failed to map write buffer" << endl; + return TestFail; + } + + MappedFrameBuffer rw_map(buffer.get(), PROT_READ | PROT_WRITE); + if (!rw_map.isValid()) { + cout << "Failed to map RW buffer" << endl; + return TestFail; + } + + return TestPass; + } + +private: + std::unique_ptr config_; + FrameBufferAllocator *allocator_; + Stream *stream_; +}; + +} /* namespace */ + +TEST_REGISTER(MappedBufferTest); diff --git a/test/meson.build b/test/meson.build index 775187159dec..0a1d434e3996 100644 --- a/test/meson.build +++ b/test/meson.build @@ -31,6 +31,7 @@ internal_tests = [ ['file', 'file.cpp'], ['file-descriptor', 'file-descriptor.cpp'], ['hotplug-cameras', 'hotplug-cameras.cpp'], + ['mapped-buffer', 'mapped-buffer.cpp'], ['message', 'message.cpp'], ['object', 'object.cpp'], ['object-delete', 'object-delete.cpp'], From patchwork Wed Aug 5 15:14:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9232 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3C3FABD86F for ; Wed, 5 Aug 2020 15:15:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4AF160730; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GKUrhLKp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6A65A60737 for ; Wed, 5 Aug 2020 17:15:14 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 63B8DDA1; Wed, 5 Aug 2020 17:15:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640512; bh=dvsmXy48aQgVz1ENpEM12XqvzUq6OSx0WuWTFauAczY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GKUrhLKpE6s1Et3MPJhK42KwQC9OQer+Q/vGF9Ui0vb2Zhtc/zRZp6v5LxRcu6q5h 3pSncDPjI8kcHJYQJ5mAKLEzU7w6FeAS0iyuJqDDRfnNhLFbIExhLBwKwbUx0+7esw eqAK4SN7NEs40svlMdrlE3xPiBTkh/ea2ZOuWvTo= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:14:59 +0100 Message-Id: <20200805151507.227503-6-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/13] libcamera: buffer: Convert copyFrom to use MappedFrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Utilise the new MappedFrameBuffer helper to handle all mapping and unmapping of the copyFrom helper function. Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- src/libcamera/buffer.cpp | 37 ++++++++++++++----------------------- 1 file changed, 14 insertions(+), 23 deletions(-) diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 8666f1992a77..a2b66dc86129 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -259,32 +259,23 @@ int FrameBuffer::copyFrom(const FrameBuffer *src) } } - for (unsigned int i = 0; i < planes_.size(); i++) { - void *dstmem = mmap(nullptr, planes_[i].length, PROT_WRITE, - MAP_SHARED, planes_[i].fd.fd(), 0); + MappedFrameBuffer source(src, PROT_READ); + MappedFrameBuffer destination(this, PROT_WRITE); - if (dstmem == MAP_FAILED) { - LOG(Buffer, Error) - << "Failed to map destination plane " << i; - metadata_.status = FrameMetadata::FrameError; - return -EINVAL; - } - - void *srcmem = mmap(nullptr, src->planes_[i].length, PROT_READ, - MAP_SHARED, src->planes_[i].fd.fd(), 0); - - if (srcmem == MAP_FAILED) { - munmap(dstmem, planes_[i].length); - LOG(Buffer, Error) - << "Failed to map source plane " << i; - metadata_.status = FrameMetadata::FrameError; - return -EINVAL; - } + if (!source.isValid()) { + LOG(Buffer, Error) << "Failed to map source planes"; + return -EINVAL; + } - memcpy(dstmem, srcmem, src->planes_[i].length); + if (!destination.isValid()) { + LOG(Buffer, Error) << "Failed to map destination planes"; + return -EINVAL; + } - munmap(srcmem, src->planes_[i].length); - munmap(dstmem, planes_[i].length); + for (unsigned int i = 0; i < planes_.size(); i++) { + memcpy(destination.maps()[i].data(), + source.maps()[i].data(), + source.maps()[i].size()); } metadata_ = src->metadata_; From patchwork Wed Aug 5 15:15:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9233 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 97DC7BD87A for ; Wed, 5 Aug 2020 15:15:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 631A4605B8; Wed, 5 Aug 2020 17:15:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P2/G3x0a"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D7FDD60737 for ; Wed, 5 Aug 2020 17:15:14 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1FBCBDAD; Wed, 5 Aug 2020 17:15:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640514; bh=Sfl69kJpIlfGIvE3OlZQiHy5W7qNX854yKeY8eANFfw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P2/G3x0aAxUM0z2RyFricFoqUbXp9f8WYSOoqGso++NH4RZ+ERCIbRgMwbbay13GD nUg/4M5kOKzyysftlGEHN1LjaO51aMAK8HmLqj04T7/wH9TQlFPCpWRF8FSN5VxxVq Vg/Icy5fm6IoaFOiBybNd42lqj6ijrMyIw3Pb0vw= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:00 +0100 Message-Id: <20200805151507.227503-7-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/13] android: camera_device: Fix trivial spelling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix the spelling of 'successfully' reported in an error message. Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- src/android/camera_device.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index c1e581262921..6235ef52df88 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1220,7 +1220,7 @@ void CameraDevice::requestComplete(Request *request) std::unique_ptr resultMetadata; if (request->status() != Request::RequestComplete) { - LOG(HAL, Error) << "Request not succesfully completed: " + LOG(HAL, Error) << "Request not successfully completed: " << request->status(); status = CAMERA3_BUFFER_STATUS_ERROR; } From patchwork Wed Aug 5 15:15:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9234 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E7E2BD87A for ; Wed, 5 Aug 2020 15:15:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE95F60392; Wed, 5 Aug 2020 17:15:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WhMjvyPD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5710260737 for ; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8C522FED; Wed, 5 Aug 2020 17:15:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640514; bh=yY0O3lcI4a/vDxUU+xyYQw0f0jKZNbE3mUORFiDzGU0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WhMjvyPDb7Y8mIB6oXNTSoNKgwt0DfhzE1vlEXDfix+7fSIQ7WnT7TvzwUiEKnBhA kXurl4X67C18RHnsaFsJgSgEtdRjuYunzSPEkWIZyCf2h2Tk/dJBx6+lVaBQSo0Y7N ADVRffQxcgraRiXpLNOvyH9dTJOGQA/s3HjklTxE= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:01 +0100 Message-Id: <20200805151507.227503-8-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/13] android: camera_device: Report an error in notifyError() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If an error is generated, try to be verbose about it in the libcamera logs. Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- v2: - Add todo to note possible debug info expansion - Fix typo - Print .toString() formatted pixel format v3: - Remove colon from todo: --- src/android/camera_device.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 6235ef52df88..60af59ce154c 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1294,6 +1294,13 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream) { camera3_notify_msg_t notify = {}; + /* + * \todo Report and identify the stream number or configuration to + * clarify the stream that failed. + */ + LOG(HAL, Error) << "Error occurred on frame " << frameNumber << " (" + << toPixelFormat(stream->format).toString() << ")"; + notify.type = CAMERA3_MSG_ERROR; notify.message.error.error_stream = stream; notify.message.error.frame_number = frameNumber; From patchwork Wed Aug 5 15:15:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9235 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7AC55BD87A for ; Wed, 5 Aug 2020 15:15:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 43C5660737; Wed, 5 Aug 2020 17:15:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DdHduWEv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A7C57607EB for ; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3985310CF; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640515; bh=Q4hfjc6Ay+uc0VMtoVvNDXBriK4jpJv7sMKBqgilycg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DdHduWEvRfkNqWUtS5bdQmtLGil22i6zNRvFv/q0YvZwrr0C0G3kpHshOXO+peKgU /4wVa42EVZgD5RAvKeGtmYusryyhfAWwgFH9fUXmOodEG7AcDw/GPEddle+K8YxU81 2NzjrfzWWeh5uSiFnLhBGsXAJ//6dNlv7Cf3XvnA= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:02 +0100 Message-Id: <20200805151507.227503-9-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/13] android: camera_device: Only construct required planes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The camera3buffer describes the number of filedescriptors given. Don't try to construct more planes than that. Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- v4: - Return nullptr on error instead of breaking --- src/android/camera_device.cpp | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 60af59ce154c..27bcf3ab1930 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1118,9 +1118,20 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) FrameBuffer *CameraDevice::createFrameBuffer(const buffer_handle_t camera3buffer) { std::vector planes; - for (unsigned int i = 0; i < 3; i++) { + for (int i = 0; i < camera3buffer->numFds; i++) { + /* Skip unused planes. */ + if (camera3buffer->data[i] == -1) + break; + FrameBuffer::Plane plane; plane.fd = FileDescriptor(camera3buffer->data[i]); + if (!plane.fd.isValid()) { + LOG(HAL, Error) << "Failed to obtain FileDescriptor (" + << camera3buffer->data[i] << ") " + << " on plane " << i; + return nullptr; + } + /* * Setting length to zero here is OK as the length is only used * to map the memory of the plane. Libcamera do not need to poke From patchwork Wed Aug 5 15:15:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9236 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E12FFBD87A for ; Wed, 5 Aug 2020 15:15:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AEB7C6088E; Wed, 5 Aug 2020 17:15:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tlSLJqaM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 236B16087E for ; Wed, 5 Aug 2020 17:15:16 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 968502C0; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640515; bh=cLrxAI8ZbDotkQjFuJKk/8/i7PqD7BKXIjPauSR67Fw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tlSLJqaMZf6KNaY/ekJUWEamdTdmuYhukpD4/9ysEfIiaCc0rWvaz+8dfEfCwZpMv 5GVl5L4C0nSgxSOIVxHeqiNxjd7WLlUsRjg0i8lhXT1p1AMCZYHOFiDyzw6zsAxaUh t47jMGVWnDxSIdkASDzDIEMbffpP3RUwtLcPwiAc= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:03 +0100 Message-Id: <20200805151507.227503-10-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/13] android: camera_device: Query plane length X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use lseek to query the length of planes where possible rather than leaving the plane.length as zero, which prevents mapping buffers for software processing. Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- v3 - Return nullptr if we fail to query plane length v2: - no longer support an invalid plane length. --- src/android/camera_device.cpp | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 27bcf3ab1930..a5c135ce03eb 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1132,12 +1132,13 @@ FrameBuffer *CameraDevice::createFrameBuffer(const buffer_handle_t camera3buffer return nullptr; } - /* - * Setting length to zero here is OK as the length is only used - * to map the memory of the plane. Libcamera do not need to poke - * at the memory content queued by the HAL. - */ - plane.length = 0; + off_t length = lseek(plane.fd.fd(), 0, SEEK_END); + if (length == -1) { + LOG(HAL, Error) << "Failed to query plane length"; + return nullptr; + } + + plane.length = length; planes.push_back(std::move(plane)); } From patchwork Wed Aug 5 15:15:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9237 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4BD7FBD87B for ; Wed, 5 Aug 2020 15:15:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1488F60901; Wed, 5 Aug 2020 17:15:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DS0CsY5f"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 871D9605C0 for ; Wed, 5 Aug 2020 17:15:16 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F3BFA304; Wed, 5 Aug 2020 17:15:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640516; bh=Zj/gCjWtOnbMkj33ENuOYRQrWswY1/duO6JA/MbRItU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DS0CsY5fg3z/gBzgzbYkWQ8SmIIlOOO6mr+QP4/Ts3fimUwKCfIRaosBECH5weeyo B85kbOqZedk89tszPFk8nScmYGiwtkhhGBMg3RulKzv29klcVKQignhePm9wgViY+C oPE8RkZSOwkDiTgSf/DEkb8iGrv9f5vYRqzdkA04= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:04 +0100 Message-Id: <20200805151507.227503-11-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/13] android: camera_device: Report configuration changes from validate() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When we call validate on a configuration, if there are any adjustments on the configuration, we fail without showing why. Display the stream configuration after the validate stage to aid debugging stream startup failures. Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- v4 - hyphenate instead. --- src/android/camera_device.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a5c135ce03eb..397161a9ab12 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1084,6 +1084,10 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) break; case CameraConfiguration::Adjusted: LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config_) + LOG(HAL, Info) << " - " << cfg.toString(); + config_.reset(); return -EINVAL; case CameraConfiguration::Invalid: From patchwork Wed Aug 5 15:15:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9238 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A84DFBD87A for ; Wed, 5 Aug 2020 15:15:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 721BA60822; Wed, 5 Aug 2020 17:15:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="U3bMjyk6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5D2460392 for ; Wed, 5 Aug 2020 17:15:16 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C88DFD1; Wed, 5 Aug 2020 17:15:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640516; bh=uU9j6xuLlGvVO0swIxkQsGpz3Ve+CqHUKQQX3Ek9C84=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U3bMjyk6oRkRPR5MMWhqpAIqJ1YIEh7OAqYyoQ5m+oCRWwPtKiOwLu+vIDVu0SLzZ lHGk1gZgaT/CjldGux+JmPQ2Ml8HA29JslXUnbUMlD+yFMxzhQRo8INHprRjo4FKJv dZ4ou2nkkEcH8y6JqqFwWFl5UbJq4yWadRC/yPsc= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:05 +0100 Message-Id: <20200805151507.227503-12-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/13] android: camera_device: Generate ResultMetadata earlier X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Generate the ResultMetadata before performing JPEG compression so that JPEG specific metadata can be added to the metadata when it has been processed. Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- src/android/camera_device.cpp | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 397161a9ab12..23db5f830a71 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1234,6 +1234,8 @@ void CameraDevice::requestComplete(Request *request) const std::map &buffers = request->buffers(); camera3_buffer_status status = CAMERA3_BUFFER_STATUS_OK; std::unique_ptr resultMetadata; + Camera3RequestDescriptor *descriptor = + reinterpret_cast(request->cookie()); if (request->status() != Request::RequestComplete) { LOG(HAL, Error) << "Request not successfully completed: " @@ -1241,9 +1243,17 @@ void CameraDevice::requestComplete(Request *request) status = CAMERA3_BUFFER_STATUS_ERROR; } + /* + * \todo The timestamp used for the metadata is currently always taken + * from the first buffer (which may be the first stream) in the Request. + * It might be appropriate to return a 'correct' (as determined by + * pipeline handlers) timestamp in the Request itself. + */ + FrameBuffer *buffer = buffers.begin()->second; + resultMetadata = getResultMetadata(descriptor->frameNumber, + buffer->metadata().timestamp); + /* Prepare to call back the Android camera stack. */ - Camera3RequestDescriptor *descriptor = - reinterpret_cast(request->cookie()); camera3_capture_result_t captureResult = {}; captureResult.frame_number = descriptor->frameNumber; @@ -1256,21 +1266,12 @@ void CameraDevice::requestComplete(Request *request) captureResult.output_buffers = const_cast(descriptor->buffers); - /* - * \todo The timestamp used for the metadata is currently always taken - * from the first buffer (which may be the first stream) in the Request. - * It might be appropriate to return a 'correct' (as determined by - * pipeline handlers) timestamp in the Request itself. - */ - FrameBuffer *buffer = buffers.begin()->second; if (status == CAMERA3_BUFFER_STATUS_OK) { notifyShutter(descriptor->frameNumber, buffer->metadata().timestamp); captureResult.partial_result = 1; - resultMetadata = getResultMetadata(descriptor->frameNumber, - buffer->metadata().timestamp); captureResult.result = resultMetadata->get(); } From patchwork Wed Aug 5 15:15:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9239 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 15B89BD87B for ; Wed, 5 Aug 2020 15:15:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CBF71608E8; Wed, 5 Aug 2020 17:15:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Q6Z5t7IK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 602FC60392 for ; Wed, 5 Aug 2020 17:15:17 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D87B011F6; Wed, 5 Aug 2020 17:15:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640517; bh=7cq7S51KXFxlSSgAB/IfwBLRFxPEnXdsH9LEfOb5HxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q6Z5t7IKTL87z9ATVFEXqYLxEYaf/+tzI8priaNdSv3nHWv1yAADxTSLGgCZfwi+R 6nkwwO2jr4ZGWjnGzfzSX/Lbu7QBefNPC0axdCbBerITSwzeg8yIRW+Z1fM4aIzoVa GtKsc8b1PuoQttvY46qIbX/LfrWo2nQWzkeF2/dg= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:06 +0100 Message-Id: <20200805151507.227503-13-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/13] android: Introduce JPEG compression X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a compressor interface and implement a JPEG compressor using libjpeg. Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- v2: - Convert to use the libcamera format information rather than duplicating it ourselves. - Not easy to get the horizontal subsampling - not easy to determine if we have an nvSwap... v3: - Fix frame.error return value (negative inversion) - Fix comments - constify data table - set jpeg image lenght to zero after free - set override on class interface functions. - Remove ununsed fucntion prototype - Rename pixelFormatInfo to pixelFormatInfo_ v4: - Add todo regarding strides from buffers - Rename to compressor_libjpeg rather than compressor_jpeg - Define android dependencies in their own variable in meson --- src/android/jpeg/compressor.h | 25 +++ src/android/jpeg/compressor_libjpeg.cpp | 217 ++++++++++++++++++++++++ src/android/jpeg/compressor_libjpeg.h | 41 +++++ src/android/meson.build | 5 + src/libcamera/meson.build | 2 + 5 files changed, 290 insertions(+) create mode 100644 src/android/jpeg/compressor.h create mode 100644 src/android/jpeg/compressor_libjpeg.cpp create mode 100644 src/android/jpeg/compressor_libjpeg.h diff --git a/src/android/jpeg/compressor.h b/src/android/jpeg/compressor.h new file mode 100644 index 000000000000..113693b8cdb6 --- /dev/null +++ b/src/android/jpeg/compressor.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * compressor.h - Image compression interface + */ +#ifndef __ANDROID_JPEG_COMPRESSOR_H__ +#define __ANDROID_JPEG_COMPRESSOR_H__ + +#include +#include +#include + +class Compressor +{ +public: + using CompressedImage = libcamera::Span; + + virtual ~Compressor() {}; + + virtual int configure(const libcamera::StreamConfiguration &cfg) = 0; + virtual int compress(const libcamera::FrameBuffer *source, const CompressedImage &image) = 0; +}; + +#endif /* __ANDROID_JPEG_COMPRESSOR_H__ */ diff --git a/src/android/jpeg/compressor_libjpeg.cpp b/src/android/jpeg/compressor_libjpeg.cpp new file mode 100644 index 000000000000..5c843ab1dd47 --- /dev/null +++ b/src/android/jpeg/compressor_libjpeg.cpp @@ -0,0 +1,217 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * compressor_libjpeg.cpp - JPEG compression using libjpeg native API + */ + +#include "compressor_libjpeg.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/log.h" + +using namespace libcamera; + +LOG_DEFINE_CATEGORY(JPEG) + +namespace { + +struct JPEGPixelFormatInfo { + J_COLOR_SPACE colorSpace; + const PixelFormatInfo &pixelFormatInfo; + bool nvSwap; +}; + +const std::map pixelInfo{ + { formats::R8, { JCS_GRAYSCALE, PixelFormatInfo::info(formats::R8), false } }, + + { formats::RGB888, { JCS_EXT_BGR, PixelFormatInfo::info(formats::RGB888), false } }, + { formats::BGR888, { JCS_EXT_RGB, PixelFormatInfo::info(formats::BGR888), false } }, + + { formats::NV12, { JCS_YCbCr, PixelFormatInfo::info(formats::NV12), false } }, + { formats::NV21, { JCS_YCbCr, PixelFormatInfo::info(formats::NV21), true } }, + { formats::NV16, { JCS_YCbCr, PixelFormatInfo::info(formats::NV16), false } }, + { formats::NV61, { JCS_YCbCr, PixelFormatInfo::info(formats::NV61), true } }, + { formats::NV24, { JCS_YCbCr, PixelFormatInfo::info(formats::NV24), false } }, + { formats::NV42, { JCS_YCbCr, PixelFormatInfo::info(formats::NV42), true } }, +}; + +const struct JPEGPixelFormatInfo &findPixelInfo(const PixelFormat &format) +{ + static const struct JPEGPixelFormatInfo invalidPixelFormat { + JCS_UNKNOWN, PixelFormatInfo(), false + }; + + const auto iter = pixelInfo.find(format); + if (iter == pixelInfo.end()) { + LOG(JPEG, Error) << "Unsupported pixel format for JPEG compressor: " + << format.toString(); + return invalidPixelFormat; + } + + return iter->second; +} + +} /* namespace */ + +CompressorLibJpeg::CompressorLibJpeg() + : quality_(95) +{ + /* \todo Expand error handling coverage with a custom handler. */ + compress_.err = jpeg_std_error(&jerr_); + + jpeg_create_compress(&compress_); +} + +CompressorLibJpeg::~CompressorLibJpeg() +{ + jpeg_destroy_compress(&compress_); +} + +int CompressorLibJpeg::configure(const StreamConfiguration &cfg) +{ + const struct JPEGPixelFormatInfo info = findPixelInfo(cfg.pixelFormat); + if (info.colorSpace == JCS_UNKNOWN) + return -ENOTSUP; + + compress_.image_width = cfg.size.width; + compress_.image_height = cfg.size.height; + compress_.in_color_space = info.colorSpace; + + compress_.input_components = info.colorSpace == JCS_GRAYSCALE ? 1 : 3; + + jpeg_set_defaults(&compress_); + jpeg_set_quality(&compress_, quality_, TRUE); + + pixelFormatInfo_ = &info.pixelFormatInfo; + + nv_ = pixelFormatInfo_->numPlanes() == 2; + nvSwap_ = info.nvSwap; + + return 0; +} + +void CompressorLibJpeg::compressRGB(const libcamera::MappedBuffer *frame) +{ + unsigned char *src = static_cast(frame->maps()[0].data()); + /* \todo Stride information should come from buffer configuration. */ + unsigned int stride = pixelFormatInfo_->stride(compress_.image_width, 0); + + JSAMPROW row_pointer[1]; + + while (compress_.next_scanline < compress_.image_height) { + row_pointer[0] = &src[compress_.next_scanline * stride]; + jpeg_write_scanlines(&compress_, row_pointer, 1); + } +} + +/* + * Compress the incoming buffer from a supported NV format. + * This naively unpacks the semi-planar NV12 to a YUV888 format for libjpeg. + */ +void CompressorLibJpeg::compressNV(const libcamera::MappedBuffer *frame) +{ + uint8_t tmprowbuf[compress_.image_width * 3]; + + /* + * \todo Use the raw api, and only unpack the cb/cr samples to new line + * buffers. If possible, see if we can set appropriate pixel strides + * too to save even that copy. + * + * Possible hints at: + * https://sourceforge.net/p/libjpeg/mailman/message/30815123/ + */ + unsigned int y_stride = pixelFormatInfo_->stride(compress_.image_width, 0); + unsigned int c_stride = pixelFormatInfo_->stride(compress_.image_width, 1); + + unsigned int horzSubSample = 2 * compress_.image_width / c_stride; + unsigned int vertSubSample = pixelFormatInfo_->planes[1].verticalSubSampling; + + unsigned int c_inc = horzSubSample == 1 ? 2 : 0; + unsigned int cb_pos = nvSwap_ ? 1 : 0; + unsigned int cr_pos = nvSwap_ ? 0 : 1; + + const unsigned char *src = static_cast(frame->maps()[0].data()); + const unsigned char *src_c = src + y_stride * compress_.image_height; + + JSAMPROW row_pointer[1]; + row_pointer[0] = &tmprowbuf[0]; + + for (unsigned int y = 0; y < compress_.image_height; y++) { + unsigned char *dst = &tmprowbuf[0]; + + const unsigned char *src_y = src + y * compress_.image_width; + const unsigned char *src_cb = src_c + (y / vertSubSample) * c_stride + cb_pos; + const unsigned char *src_cr = src_c + (y / vertSubSample) * c_stride + cr_pos; + + for (unsigned int x = 0; x < compress_.image_width; x += 2) { + dst[0] = *src_y; + dst[1] = *src_cb; + dst[2] = *src_cr; + src_y++; + src_cb += c_inc; + src_cr += c_inc; + dst += 3; + + dst[0] = *src_y; + dst[1] = *src_cb; + dst[2] = *src_cr; + src_y++; + src_cb += 2; + src_cr += 2; + dst += 3; + } + + jpeg_write_scanlines(&compress_, row_pointer, 1); + } +} + +int CompressorLibJpeg::compress(const FrameBuffer *source, const CompressedImage &dest) +{ + MappedFrameBuffer frame(source, PROT_READ); + if (!frame.isValid()) { + LOG(JPEG, Error) << "Failed to map FrameBuffer : " + << strerror(frame.error()); + return frame.error(); + } + + unsigned char *destination = dest.data(); + unsigned long size = dest.size(); + + /* + * The jpeg_mem_dest will reallocate if the required size is not + * sufficient. That means the output won't be written to the correct + * buffers. + * + * \todo Implement our own custom memory destination to prevent + * reallocation and prefer failure with correct reporting. + */ + jpeg_mem_dest(&compress_, &destination, &size); + + jpeg_start_compress(&compress_, TRUE); + + LOG(JPEG, Debug) << "JPEG Compress Starting:" << compress_.image_width + << "x" << compress_.image_height; + + if (nv_) + compressNV(&frame); + else + compressRGB(&frame); + + jpeg_finish_compress(&compress_); + + return size; +} diff --git a/src/android/jpeg/compressor_libjpeg.h b/src/android/jpeg/compressor_libjpeg.h new file mode 100644 index 000000000000..ba4650832332 --- /dev/null +++ b/src/android/jpeg/compressor_libjpeg.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * compressor_libjpeg.h - JPEG compression using libjpeg + */ +#ifndef __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__ +#define __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__ + +#include "compressor.h" + +#include "libcamera/internal/buffer.h" +#include "libcamera/internal/formats.h" + +#include + +class CompressorLibJpeg : public Compressor +{ +public: + CompressorLibJpeg(); + ~CompressorLibJpeg(); + + int configure(const libcamera::StreamConfiguration &cfg) override; + int compress(const libcamera::FrameBuffer *source, const CompressedImage &jpeg) override; + +private: + void compressRGB(const libcamera::MappedBuffer *frame); + void compressNV(const libcamera::MappedBuffer *frame); + + struct jpeg_compress_struct compress_; + struct jpeg_error_mgr jerr_; + + unsigned int quality_; + + const libcamera::PixelFormatInfo *pixelFormatInfo_; + + bool nv_; + bool nvSwap_; +}; + +#endif /* __ANDROID_JPEG_COMPRESSOR_LIBJPEG_H__ */ diff --git a/src/android/meson.build b/src/android/meson.build index 822cad621f01..7dd375e73eff 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -6,12 +6,17 @@ android_hal_sources = files([ 'camera_device.cpp', 'camera_metadata.cpp', 'camera_ops.cpp', + 'jpeg/compressor_libjpeg.cpp', ]) android_camera_metadata_sources = files([ 'metadata/camera_metadata.c', ]) +android_deps = [ + dependency('libjpeg'), +] + android_camera_metadata = static_library('camera_metadata', android_camera_metadata_sources, include_directories : android_includes) diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 3aad4386ffc2..8ad0a1b0740f 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -124,6 +124,8 @@ if get_option('android') libcamera_sources += android_hal_sources includes += android_includes libcamera_link_with += android_camera_metadata + + libcamera_deps += android_deps endif # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag. From patchwork Wed Aug 5 15:15:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9240 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6D9C7BD87A for ; Wed, 5 Aug 2020 15:15:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3EA3460905; Wed, 5 Aug 2020 17:15:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LxSv3m3M"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B687060392 for ; Wed, 5 Aug 2020 17:15:17 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 45A5011FF; Wed, 5 Aug 2020 17:15:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1596640517; bh=ldFBnyzQt9QgKplOjiqJam1m9MKfJtQNClumlzOUhOU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LxSv3m3MsP9WnENY7ec2IeovuYvRS38I0ECKB0BYsfDDgFJSZkO4oYkD3Qa2AtoFo jX7Uy7G4Oaiiv9SAJ0uj+v2R7xfDCrI+PrNoicpo/Ztuevbx/N1xEPoy1eA8YMytAs 724xIGKCfN5epDUYKnNpxjmt2urpgo/F/PIREGe0= From: Kieran Bingham To: libcamera devel Date: Wed, 5 Aug 2020 16:15:07 +0100 Message-Id: <20200805151507.227503-14-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> References: <20200805151507.227503-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 13/13] android: camera_device: Support MJPEG stream construction X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" MJPEG streams must be created referencing a libcamera stream. This stream may already be provided by the request configuration, in which case the existing stream is utilised. If no compatible stream is available to encode, a new stream is requested from the libcamera configuration. Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- v3 - Add max jpeg size todo - Fix metadata allocations - Use a class member to store the max jpeg buffer size - Remove the scoping level for jpeg blob header - Don't rely on the compressor pointer as a flag - Fix camera metadata size allocations v4: - add an error path when no compressor is available (but expected) - Rebase on top of Jacopo's metadata changes (new metadata counts) --- src/android/camera_device.cpp | 230 ++++++++++++++++++++++++++++++++-- src/android/camera_device.h | 12 ++ 2 files changed, 233 insertions(+), 9 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 23db5f830a71..c182bef046c7 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -8,6 +8,7 @@ #include "camera_device.h" #include "camera_ops.h" +#include #include #include @@ -22,6 +23,8 @@ #include "camera_metadata.h" #include "system/graphics.h" +#include "jpeg/compressor_libjpeg.h" + using namespace libcamera; namespace { @@ -129,6 +132,52 @@ const std::map camera3FormatsMap = { LOG_DECLARE_CATEGORY(HAL); +class MappedCamera3Buffer : public MappedBuffer +{ +public: + MappedCamera3Buffer(const buffer_handle_t camera3buffer, int flags); +}; + +MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer, + int flags) +{ + maps_.reserve(camera3buffer->numFds); + error_ = 0; + + for (int i = 0; i < camera3buffer->numFds; i++) { + if (camera3buffer->data[i] == -1) + continue; + + off_t length = lseek(camera3buffer->data[i], 0, SEEK_END); + if (length < 0) { + error_ = -errno; + LOG(HAL, Error) << "Failed to query plane length"; + break; + } + + void *address = mmap(nullptr, length, flags, MAP_SHARED, + camera3buffer->data[i], 0); + if (address == MAP_FAILED) { + error_ = -errno; + LOG(HAL, Error) << "Failed to mmap plane"; + break; + } + + maps_.emplace_back(static_cast(address), + static_cast(length)); + } +} + +CameraStream::CameraStream(PixelFormat f, Size s) + : index(-1), format(f), size(s), jpeg(nullptr) +{ +} + +CameraStream::~CameraStream() +{ + delete jpeg; +}; + /* * \struct Camera3RequestDescriptor * @@ -167,6 +216,12 @@ CameraDevice::CameraDevice(unsigned int id, const std::shared_ptr &camer facing_(CAMERA_FACING_FRONT), orientation_(0) { camera_->requestCompleted.connect(this, &CameraDevice::requestComplete); + + /* + * \todo Determine a more accurate value for this during + * streamConfiguration. + */ + maxJpegBufferSize_ = 13 << 20; /* 13631488 from USB HAL */ } CameraDevice::~CameraDevice() @@ -417,10 +472,10 @@ std::tuple CameraDevice::calculateStaticMetadataSize() { /* * \todo Keep this in sync with the actual number of entries. - * Currently: 50 entries, 671 bytes of static metadata + * Currently: 51 entries, 687 bytes of static metadata */ - uint32_t numEntries = 50; - uint32_t byteSize = 671; + uint32_t numEntries = 51; + uint32_t byteSize = 687; /* * Calculate space occupation in bytes for dynamically built metadata @@ -576,6 +631,12 @@ const camera_metadata_t *CameraDevice::getStaticMetadata() availableThumbnailSizes.data(), availableThumbnailSizes.size()); + /* + * \todo Calculate the maximum JPEG buffer size by asking the compressor + * giving the maximum frame size required. + */ + staticMetadata_->addEntry(ANDROID_JPEG_MAX_SIZE, &maxJpegBufferSize_, 1); + /* Sensor static metadata. */ int32_t pixelArraySize[] = { 2592, 1944, @@ -789,6 +850,7 @@ const camera_metadata_t *CameraDevice::getStaticMetadata() ANDROID_CONTROL_AWB_LOCK_AVAILABLE, ANDROID_CONTROL_AVAILABLE_MODES, ANDROID_JPEG_AVAILABLE_THUMBNAIL_SIZES, + ANDROID_JPEG_MAX_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, ANDROID_SENSOR_INFO_ACTIVE_ARRAY_SIZE, ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, @@ -860,6 +922,9 @@ const camera_metadata_t *CameraDevice::getStaticMetadata() ANDROID_SENSOR_EXPOSURE_TIME, ANDROID_STATISTICS_LENS_SHADING_MAP_MODE, ANDROID_STATISTICS_SCENE_FLICKER, + ANDROID_JPEG_SIZE, + ANDROID_JPEG_QUALITY, + ANDROID_JPEG_ORIENTATION, }; staticMetadata_->addEntry(ANDROID_REQUEST_AVAILABLE_RESULT_KEYS, availableResultKeys.data(), @@ -1052,8 +1117,10 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) */ unsigned int streamIndex = 0; + /* First handle all non-MJPEG streams. */ for (unsigned int i = 0; i < stream_list->num_streams; ++i) { camera3_stream_t *stream = stream_list->streams[i]; + Size size(stream->width, stream->height); PixelFormat format = toPixelFormat(stream->format); @@ -1067,16 +1134,71 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) if (!format.isValid()) return -EINVAL; + streams_.emplace_back(format, size); + stream->priv = static_cast(&streams_[i]); + + /* Defer handling of MJPEG streams until all others are known. */ + if (format == formats::MJPEG) + continue; + StreamConfiguration streamConfiguration; - streamConfiguration.size.width = stream->width; - streamConfiguration.size.height = stream->height; + streamConfiguration.size = size; streamConfiguration.pixelFormat = format; config_->addConfiguration(streamConfiguration); streams_[i].index = streamIndex++; - stream->priv = static_cast(&streams_[i]); + } + + /* Now handle MJPEG streams, adding a new stream if required. */ + for (unsigned int i = 0; i < stream_list->num_streams; ++i) { + camera3_stream_t *stream = stream_list->streams[i]; + bool match = false; + + if (streams_[i].format != formats::MJPEG) + continue; + + /* Search for a compatible stream */ + for (unsigned int j = 0; j < config_->size(); j++) { + StreamConfiguration &cfg = config_->at(j); + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size == streams_[i].size) { + LOG(HAL, Info) << "Stream " << i + << " using libcamera stream " << j; + + match = true; + streams_[i].index = j; + } + } + + /* + * Without a compatible match for JPEG encoding we must + * introduce a new stream to satisfy the request requirements. + */ + if (!match) { + StreamConfiguration streamConfiguration; + + /* + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. + */ + streamConfiguration.size.width = stream->width; + streamConfiguration.size.height = stream->height; + streamConfiguration.pixelFormat = formats::NV12; + + LOG(HAL, Info) << "Adding " << streamConfiguration.toString() + << " for MJPEG support"; + + config_->addConfiguration(streamConfiguration); + streams_[i].index = streamIndex++; + } } switch (config_->validate()) { @@ -1103,6 +1225,20 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) /* Use the bufferCount confirmed by the validation process. */ stream->max_buffers = cfg.bufferCount; + + /* + * Construct a software compressor for MJPEG streams from the + * chosen libcamera source stream. + */ + if (cameraStream->format == formats::MJPEG) { + cameraStream->jpeg = new CompressorLibJpeg(); + int ret = cameraStream->jpeg->configure(cfg); + if (ret) { + LOG(HAL, Error) + << "Failed to configure compressor"; + return ret; + } + } } /* @@ -1197,6 +1333,10 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques descriptor->buffers[i].stream = camera3Buffers[i].stream; descriptor->buffers[i].buffer = camera3Buffers[i].buffer; + /* Software streams are handled after hardware streams complete. */ + if (cameraStream->format == formats::MJPEG) + continue; + /* * Create a libcamera buffer using the dmabuf descriptors of * the camera3Buffer for each stream. The FrameBuffer is @@ -1253,8 +1393,80 @@ void CameraDevice::requestComplete(Request *request) resultMetadata = getResultMetadata(descriptor->frameNumber, buffer->metadata().timestamp); - /* Prepare to call back the Android camera stack. */ + /* Handle any JPEG compression. */ + for (unsigned int i = 0; i < descriptor->numBuffers; ++i) { + CameraStream *cameraStream = + static_cast(descriptor->buffers[i].stream->priv); + + if (cameraStream->format != formats::MJPEG) + continue; + + Compressor *compressor = cameraStream->jpeg; + if (!compressor) { + LOG(HAL, Error) << "Failed to identify compressor"; + continue; + } + StreamConfiguration *streamConfiguration = &config_->at(cameraStream->index); + Stream *stream = streamConfiguration->stream(); + FrameBuffer *buffer = request->findBuffer(stream); + if (!buffer) { + LOG(HAL, Error) << "Failed to find a source stream buffer"; + continue; + } + + /* + * \todo Buffer mapping and compression should be moved to a + * separate thread. + */ + + MappedCamera3Buffer mapped(*descriptor->buffers[i].buffer, + PROT_READ | PROT_WRITE); + if (!mapped.isValid()) { + LOG(HAL, Error) << "Failed to mmap android blob buffer"; + continue; + } + + int jpeg_size = compressor->compress(buffer, mapped.maps()[0]); + if (jpeg_size < 0) { + LOG(HAL, Error) << "Failed to compress stream image"; + status = CAMERA3_BUFFER_STATUS_ERROR; + continue; + } + + /* + * Fill in the JPEG blob header. + * + * The mapped size of the buffer is being returned as + * substantially larger than the requested JPEG_MAX_SIZE + * (which is referenced from maxJpegBufferSize_). Utilise + * this static size to ensure the correct offset of the blob is + * determined. + * + * \todo Investigate if the buffer size mismatch is an issue or + * expected behaviour. + */ + uint8_t *resultPtr = mapped.maps()[0].data() + + maxJpegBufferSize_ - + sizeof(struct camera3_jpeg_blob); + auto *blob = reinterpret_cast(resultPtr); + blob->jpeg_blob_id = CAMERA3_JPEG_BLOB_ID; + blob->jpeg_size = jpeg_size; + + /* Update the JPEG result Metadata. */ + resultMetadata->addEntry(ANDROID_JPEG_SIZE, + &jpeg_size, 1); + + const uint32_t jpeg_quality = 95; + resultMetadata->addEntry(ANDROID_JPEG_QUALITY, + &jpeg_quality, 1); + + const uint32_t jpeg_orientation = 0; + resultMetadata->addEntry(ANDROID_JPEG_ORIENTATION, + &jpeg_orientation, 1); + } + + /* Prepare to call back the Android camera stack. */ camera3_capture_result_t captureResult = {}; captureResult.frame_number = descriptor->frameNumber; captureResult.num_output_buffers = descriptor->numBuffers; @@ -1334,10 +1546,10 @@ std::unique_ptr CameraDevice::getResultMetadata(int frame_number { /* * \todo Keep this in sync with the actual number of entries. - * Currently: 12 entries, 36 bytes + * Currently: 18 entries, 62 bytes */ std::unique_ptr resultMetadata = - std::make_unique(15, 50); + std::make_unique(18, 62); if (!resultMetadata->isValid()) { LOG(HAL, Error) << "Failed to allocate static metadata"; return nullptr; diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 4e5fb98c9a07..87cb939a875c 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -23,15 +23,25 @@ #include "libcamera/internal/log.h" #include "libcamera/internal/message.h" +#include "jpeg/compressor.h" + class CameraMetadata; struct CameraStream { + CameraStream(libcamera::PixelFormat, libcamera::Size); + ~CameraStream(); + /* * The index of the libcamera StreamConfiguration as added during * configureStreams(). A single libcamera Stream may be used to deliver * one or more streams to the Android framework. */ unsigned int index; + + libcamera::PixelFormat format; + libcamera::Size size; + + Compressor *jpeg; }; class CameraDevice : protected libcamera::Loggable @@ -105,6 +115,8 @@ private: int facing_; int orientation_; + + unsigned int maxJpegBufferSize_; }; #endif /* __ANDROID_CAMERA_DEVICE_H__ */