From patchwork Fri Sep 18 15:20:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomi Valkeinen X-Patchwork-Id: 9671 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A8DEC3B5D for ; Fri, 18 Sep 2020 15:21:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E5DF362FCD; Fri, 18 Sep 2020 17:21:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="qV8sDXVM"; dkim-atps=neutral Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7411B62FAB for ; Fri, 18 Sep 2020 17:21:01 +0200 (CEST) Received: by mail-lf1-x136.google.com with SMTP id d15so6511496lfq.11 for ; Fri, 18 Sep 2020 08:21:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1E1P1JfouIvoqAcSQcz8mxEV25Kx5h+CZg32vriOSpY=; b=qV8sDXVMUf3mfv7Oqjk/UJNowN9u628qXbnEHmoNHTI8z2s1oPp2Nxkw3UhHoWWa4e wXqhqwRVwt+Foo7355MwM7iE6LciWAlxDcHYa8VTPBLBZ8Ll7n2O1ZD8jW6Mw3Ju65KJ oS2SMxwuM44c5t859GpxwSMNP1CC+3+IXI+GuLvH82rn1y6NUEfG6ZomDGEXT9+Q4I7S /GKwcTXcqchV8tdrMmhs5DvjMWWXQ1Kjbi8eEtp7FxSA1yVeRAlCTDD9aVH7c2rMzvCf NKiRIJslvuQPi6ej40T3ICqGjmazhHqcFpYeoNcqPFPTMH5E8j2aGRmV64qm4f4caXXd XgGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=1E1P1JfouIvoqAcSQcz8mxEV25Kx5h+CZg32vriOSpY=; b=LW2OJz7JKSA969AvcFquHa2OIrKU8cWsvEqdr2PWR6ElIV4y0nbW6rgdsLL3cak5ji 0riDSyZOmAA62GhL0R21QeKywiDpmQst1mQjpV2iqWaiSKXpeBB09ccuWL6i/4CkZyZg 7T8iUpD9oG2+8qj5Meuh93vapj0++LJ1gauCb260pkonbgdjfOzIH3tAdk6lIV6jOWyp TVIEKHrtvBtgUGT6NCuGHB6UkCBKAqA0B3kXnIpZYxEs7XfRukk2Kenuj4W8YEv+jrGE 8IkUEtIO3eyUtu76TU8reIEJys/YW/BnemKj0O/OGi5vhzf813g5+TxvQn+D8fOoB7J/ TKlw== X-Gm-Message-State: AOAM531HYelRHwS6i1H9nDvYQyQ2MYxCahnIN+mwxLePy8OfK+ADh7pN +16EmvD66asmBu2VgXzZY7Io8Lvg07Ytmg== X-Google-Smtp-Source: ABdhPJyp2/BRuDvRBCqWi9hQEZ2tWCTBSdQ+pb1a5MZWLE8eswextC1isIre11DB8avLEKGjBaFbvg== X-Received: by 2002:a05:6512:1051:: with SMTP id c17mr12286412lfb.20.1600442460389; Fri, 18 Sep 2020 08:21:00 -0700 (PDT) Received: from deskari.ti.com (91-152-83-50.elisa-laajakaista.fi. [91.152.83.50]) by smtp.gmail.com with ESMTPSA id i63sm666472lji.66.2020.09.18.08.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 08:20:59 -0700 (PDT) From: Tomi Valkeinen To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 18:20:19 +0300 Message-Id: <20200918152019.784315-5-tomi.valkeinen@iki.fi> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918152019.784315-1-tomi.valkeinen@iki.fi> References: <20200918152019.784315-1-tomi.valkeinen@iki.fi> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC 4/4] libcamera python bindings X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tomi Valkeinen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Main issues: - Memory management in general. Who owns the object, how to pass ownership, etc. - Specifically, Request is currently broken. We can't, afaik, pass ownership around. So currently Python never frees a Request, and if the Request is not given to Camera::queueRequest, it will leak. - The forced threading causes some headache. Need to take care to use gil_scoped_release when C++ context can invoke callbacks, and gil_scoped_acquire at the invoke wrapper. - Callbacks. Difficult to attach context to the callbacks. I solved it with BoundMethodFunction and using lambda captures - Need public Camera destructor. It is not clear to me why C++ allows it to be private, but pybind11 doesn't. Signed-off-by: Tomi Valkeinen --- meson.build | 1 + meson_options.txt | 2 + py/meson.build | 1 + py/pycamera/__init__.py | 29 ++++++ py/pycamera/meson.build | 35 +++++++ py/pycamera/pymain.cpp | 169 +++++++++++++++++++++++++++++++ py/test/run-valgrind.sh | 3 + py/test/run.sh | 3 + py/test/test.py | 177 +++++++++++++++++++++++++++++++++ py/test/valgrind-pycamera.supp | 17 ++++ 10 files changed, 437 insertions(+) create mode 100644 py/meson.build create mode 100644 py/pycamera/__init__.py create mode 100644 py/pycamera/meson.build create mode 100644 py/pycamera/pymain.cpp create mode 100755 py/test/run-valgrind.sh create mode 100755 py/test/run.sh create mode 100755 py/test/test.py create mode 100644 py/test/valgrind-pycamera.supp diff --git a/meson.build b/meson.build index c58d458..3d1c797 100644 --- a/meson.build +++ b/meson.build @@ -104,6 +104,7 @@ libcamera_includes = include_directories('include') subdir('include') subdir('src') subdir('utils') +subdir('py') # The documentation and test components are optional and can be disabled # through configuration values. They are enabled by default. diff --git a/meson_options.txt b/meson_options.txt index d2e07ef..45b88b6 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -32,3 +32,5 @@ option('v4l2', type : 'boolean', value : false, description : 'Compile the V4L2 compatibility layer') + +option('pycamera', type : 'feature', value : 'auto') diff --git a/py/meson.build b/py/meson.build new file mode 100644 index 0000000..42ffa22 --- /dev/null +++ b/py/meson.build @@ -0,0 +1 @@ +subdir('pycamera') diff --git a/py/pycamera/__init__.py b/py/pycamera/__init__.py new file mode 100644 index 0000000..c37571b --- /dev/null +++ b/py/pycamera/__init__.py @@ -0,0 +1,29 @@ +from .pycamera import * +from enum import Enum +import os +import struct +import mmap + +# Add a wrapper which returns an array of Cameras, which have keep-alive to the CameraManager +def __CameraManager__cameras(self): + cameras = [] + for i in range(self.num_cameras): + cameras.append(self.at(i)) + return cameras + + +CameraManager.cameras = property(__CameraManager__cameras) + +# Add a wrapper which returns an array of buffers, which have keep-alive to the FB allocator +def __FrameBufferAllocator__buffers(self, stream): + buffers = [] + for i in range(self.num_buffers(stream)): + buffers.append(self.at(stream, i)) + return buffers + +FrameBufferAllocator.buffers = __FrameBufferAllocator__buffers + +def __FrameBuffer__mmap(self, plane): + return mmap.mmap(self.fd(plane), self.length(plane), mmap.MAP_SHARED, mmap.PROT_READ) + +FrameBuffer.mmap = __FrameBuffer__mmap diff --git a/py/pycamera/meson.build b/py/pycamera/meson.build new file mode 100644 index 0000000..50bdfb8 --- /dev/null +++ b/py/pycamera/meson.build @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: CC0-1.0 + +py3_dep = dependency('python3', required : get_option('pycamera')) + +if py3_dep.found() == false + subdir_done() +endif + +pycamera_sources = files([ + 'pymain.cpp', +]) + +pycamera_deps = [ + libcamera_dep, + py3_dep, +] + +includes = [ + '../../ext/pybind11/include', +] + +destdir = get_option('libdir') + '/python' + py3_dep.version() + '/site-packages/pycamera' + +pycamera = shared_module('pycamera', + pycamera_sources, + install : true, + install_dir : destdir, + name_prefix : '', + include_directories : includes, + dependencies : pycamera_deps) + +# Copy __init__.py to build dir so that we can run without installing +configure_file(input: '__init__.py', output: '__init__.py', copy: true) + +install_data(['__init__.py'], install_dir: destdir) diff --git a/py/pycamera/pymain.cpp b/py/pycamera/pymain.cpp new file mode 100644 index 0000000..569423a --- /dev/null +++ b/py/pycamera/pymain.cpp @@ -0,0 +1,169 @@ +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include + +namespace py = pybind11; + +using namespace std; +using namespace libcamera; + +PYBIND11_MODULE(pycamera, m) { + m.def("sleep", [](double s) { + py::gil_scoped_release release; + this_thread::sleep_for(std::chrono::duration(s)); + }); + + py::class_(m, "CameraManager") + // Call cm->start implicitly, as we can't use stop() either + .def(py::init([]() { + auto cm = make_unique(); + cm->start(); + return cm; + })) + + //.def("start", &CameraManager::start) + + // stop() cannot be called, as CameraManager expects all Camera instances to be released before calling stop + // and we can't have such requirement in python, especially as we have a keep-alive from Camera to CameraManager. + // So we rely on GC and the keep-alives. + //.def("stop", &CameraManager::stop) + + .def_property_readonly("num_cameras", [](CameraManager& cm) { return cm.cameras().size(); }) + .def("at", [](CameraManager& cm, unsigned int idx) { return cm.cameras()[idx]; }, py::keep_alive<0, 1>()) + ; + + py::class_>(m, "Camera") + .def_property_readonly("id", &Camera::id) + .def("acquire", &Camera::acquire) + .def("release", &Camera::release) + .def("start", &Camera::start) + .def("stop", [](shared_ptr& self) { + // Camera::stop can cause callbacks to be invoked, so we must release GIL + py::gil_scoped_release release; + self->stop(); + }) + .def("generateConfiguration", &Camera::generateConfiguration) + .def("configure", &Camera::configure) + + // XXX created requests MUST be queued to be freed, python will not free them + .def("createRequest", &Camera::createRequest, py::arg("cookie") = 0, py::return_value_policy::reference_internal) + .def("queueRequest", &Camera::queueRequest) + + .def_property("requestCompleted", + nullptr, + [](shared_ptr& self, function f) { + if (f) { + self->requestCompleted.connect(function([f = move(f)](Request* req) { + // Called from libcamera's internal thread, so need to get GIL + py::gil_scoped_acquire acquire; + f(req); + })); + } else { + // XXX Disconnects all, as we have no means to disconnect the specific std::function + self->requestCompleted.disconnect(); + } + } + ) + + .def_property("bufferCompleted", + nullptr, + [](shared_ptr& self, function f) { + if (f) { + self->bufferCompleted.connect(function([f = move(f)](Request* req, FrameBuffer* fb) { + // Called from libcamera's internal thread, so need to get GIL + py::gil_scoped_acquire acquire; + f(req, fb); + })); + } else { + // XXX Disconnects all, as we have no means to disconnect the specific std::function + self->bufferCompleted.disconnect(); + } + } + ) + + ; + + py::class_(m, "CameraConfiguration") + .def("at", (StreamConfiguration& (CameraConfiguration::*)(unsigned int))&CameraConfiguration::at, + py::return_value_policy::reference_internal) + .def("validate", &CameraConfiguration::validate) + .def_property_readonly("size", &CameraConfiguration::size) + .def_property_readonly("empty", &CameraConfiguration::empty) + ; + + py::class_(m, "StreamConfiguration") + .def("toString", &StreamConfiguration::toString) + .def_property_readonly("stream", &StreamConfiguration::stream, + py::return_value_policy::reference_internal) + .def_property("width", + [](StreamConfiguration& c) { return c.size.width; }, + [](StreamConfiguration& c, unsigned int w) { c.size.width = w; } + ) + .def_property("height", + [](StreamConfiguration& c) { return c.size.height; }, + [](StreamConfiguration& c, unsigned int h) { c.size.height = h; } + ) + .def_property("fmt", + [](StreamConfiguration& c) { return c.pixelFormat.toString(); }, + [](StreamConfiguration& c, string fmt) { c.pixelFormat = PixelFormat::fromString(fmt); } + ) + ; + + py::enum_(m, "StreamRole") + .value("StillCapture", StreamRole::StillCapture) + .value("StillCaptureRaw", StreamRole::StillCaptureRaw) + .value("VideoRecording", StreamRole::VideoRecording) + .value("Viewfinder", StreamRole::Viewfinder) + ; + + py::class_(m, "FrameBufferAllocator") + .def(py::init>(), py::keep_alive<1, 2>()) + .def("allocate", &FrameBufferAllocator::allocate) + .def("free", &FrameBufferAllocator::free) + .def("num_buffers", [](FrameBufferAllocator& fa, Stream* stream) { return fa.buffers(stream).size(); }) + .def("at", [](FrameBufferAllocator& fa, Stream* stream, unsigned int idx) { return fa.buffers(stream).at(idx).get(); }, + py::return_value_policy::reference_internal) + ; + + py::class_>(m, "FrameBuffer") + .def_property_readonly("metadata", &FrameBuffer::metadata, py::return_value_policy::reference_internal) + .def("length", [](FrameBuffer& fb, uint32_t idx) { + const FrameBuffer::Plane &plane = fb.planes()[idx]; + return plane.length; + }) + .def("fd", [](FrameBuffer& fb, uint32_t idx) { + const FrameBuffer::Plane &plane = fb.planes()[idx]; + return plane.fd.fd(); + }) + ; + + py::class_>(m, "Stream") + ; + + py::class_>(m, "Request") + .def("addBuffer", &Request::addBuffer) + .def_property_readonly("status", &Request::status) + .def_property_readonly("buffers", &Request::buffers) + ; + + + py::enum_(m, "RequestStatus") + .value("Pending", Request::RequestPending) + .value("Complete", Request::RequestComplete) + .value("Cancelled", Request::RequestCancelled) + ; + + py::class_(m, "FrameMetadata") + .def_property_readonly("sequence", [](FrameMetadata& data) { return data.sequence; }) + .def("bytesused", [](FrameMetadata& data, uint32_t idx) { return data.planes[idx].bytesused; }) + ; +} diff --git a/py/test/run-valgrind.sh b/py/test/run-valgrind.sh new file mode 100755 index 0000000..623188e --- /dev/null +++ b/py/test/run-valgrind.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +PYTHONMALLOC=malloc PYTHONPATH=../../build/debug/py valgrind --suppressions=valgrind-pycamera.supp --leak-check=full --show-leak-kinds=definite --gen-suppressions=yes python3 test.py $* diff --git a/py/test/run.sh b/py/test/run.sh new file mode 100755 index 0000000..035d3ea --- /dev/null +++ b/py/test/run.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +PYTHONPATH=../../build/debug/py python3 test.py $* diff --git a/py/test/test.py b/py/test/test.py new file mode 100755 index 0000000..0f874d3 --- /dev/null +++ b/py/test/test.py @@ -0,0 +1,177 @@ +#!/usr/bin/python3 + +import pycamera as pycam +import time +import binascii +import argparse + +parser = argparse.ArgumentParser() +parser.add_argument("-n", "--num-frames", type=int, default=10) +parser.add_argument("-c", "--print-crc", action="store_true") +parser.add_argument("-s", "--save-frames", action="store_true") +parser.add_argument("-m", "--max-cameras", type=int) +args = parser.parse_args() + +cm = pycam.CameraManager() + +cameras = cm.cameras + +if len(cameras) == 0: + print("No cameras") + exit(0) + +print("Cameras:") +for c in cameras: + print(" {}".format(c.id)) + +contexts = [] + +for i in range(len(cameras)): + contexts.append({ "camera": cameras[i], "id": i }) + if args.max_cameras and args.max_cameras - 1 == i: + break + +for ctx in contexts: + ctx["camera"].acquire() + +def configure_camera(ctx): + camera = ctx["camera"] + + # Configure + + config = camera.generateConfiguration([pycam.StreamRole.Viewfinder]) + stream_config = config.at(0) + + #stream_config.width = 160; + #stream_config.height = 120; + #stream_config.fmt = "YUYV" + + print("Cam {}: stream config {}".format(ctx["id"], stream_config.toString())) + + camera.configure(config); + + # Allocate buffers + + stream = stream_config.stream + + allocator = pycam.FrameBufferAllocator(camera); + ret = allocator.allocate(stream) + if ret < 0: + print("Can't allocate buffers") + exit(-1) + + allocated = allocator.num_buffers(stream) + print("Cam {}: Allocated {} buffers for stream".format(ctx["id"], allocated)) + + # Create Requests + + requests = [] + buffers = allocator.buffers(stream) + + for buffer in buffers: + request = camera.createRequest() + if request == None: + print("Can't create request") + exit(-1) + + ret = request.addBuffer(stream, buffer) + if ret < 0: + print("Can't set buffer for request") + exit(-1) + + requests.append(request) + + ctx["allocator"] = allocator + ctx["requests"] = requests + + +def buffer_complete_cb(ctx, req, fb): + print("Cam {}: Buf {} Complete: {}".format(ctx["id"], ctx["bufs_completed"], req.status)) + + with fb.mmap(0) as b: + if args.print_crc: + crc = binascii.crc32(b) + print("Cam {}: CRC {:#x}".format(ctx["id"], crc)) + + if args.save_frames: + id = ctx["id"] + num = ctx["bufs_completed"] + filename = "frame-{}-{}.data".format(id, num) + with open(filename, "wb") as f: + f.write(b) + print("Cam {}: Saved {}".format(ctx["id"], filename)) + + ctx["bufs_completed"] += 1 + +def req_complete_cb(ctx, req): + camera = ctx["camera"] + + print("Cam {}: Req {} Complete: {}".format(ctx["id"], ctx["reqs_completed"], req.status)) + + bufs = req.buffers + for stream, fb in bufs.items(): + meta = fb.metadata + print("Cam {}: Buf seq {}, bytes {}".format(ctx["id"], meta.sequence, meta.bytesused(0))) + + ctx["reqs_completed"] += 1 + + if ctx["reqs_queued"] < args.num_frames: + request = camera.createRequest() + if request == None: + print("Can't create request") + exit(-1) + + for stream, fb in bufs.items(): + ret = request.addBuffer(stream, fb) + if ret < 0: + print("Can't set buffer for request") + exit(-1) + + camera.queueRequest(request) + ctx["reqs_queued"] += 1 + + +def setup_callbacks(ctx): + camera = ctx["camera"] + + ctx["reqs_queued"] = 0 + ctx["reqs_completed"] = 0 + ctx["bufs_completed"] = 0 + + camera.requestCompleted = lambda req, ctx = ctx: req_complete_cb(ctx, req) + camera.bufferCompleted = lambda req, fb, ctx = ctx: buffer_complete_cb(ctx, req, fb) + +def queue_requests(ctx): + camera = ctx["camera"] + requests = ctx["requests"] + + camera.start() + + for request in requests: + camera.queueRequest(request) + ctx["reqs_queued"] += 1 + + + +for ctx in contexts: + configure_camera(ctx) + setup_callbacks(ctx) + +for ctx in contexts: + queue_requests(ctx) + + +print("Processing...") + +# Need to release GIL here, so that callbacks can be called +while any(ctx["reqs_completed"] < args.num_frames for ctx in contexts): + pycam.sleep(0.1) + +print("Exiting...") + +for ctx in contexts: + camera = ctx["camera"] + camera.stop() + camera.release() + +print("Done") diff --git a/py/test/valgrind-pycamera.supp b/py/test/valgrind-pycamera.supp new file mode 100644 index 0000000..98c693f --- /dev/null +++ b/py/test/valgrind-pycamera.supp @@ -0,0 +1,17 @@ +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:_Znwm + fun:_ZN8pybind116moduleC1EPKcS2_ + fun:PyInit_pycamera + fun:_PyImport_LoadDynamicModuleWithSpec + obj:/usr/bin/python3.8 + obj:/usr/bin/python3.8 + fun:PyVectorcall_Call + fun:_PyEval_EvalFrameDefault + fun:_PyEval_EvalCodeWithName + fun:_PyFunction_Vectorcall + fun:_PyEval_EvalFrameDefault + fun:_PyFunction_Vectorcall +}