[{"id":21693,"web_url":"https://patchwork.libcamera.org/comment/21693/","msgid":"<163904421518.2066819.4960395426215054722@Monstersaurus>","date":"2021-12-09T10:03:35","subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Tomi Valkeinen (2021-12-09 09:29:06)\n> Add cam.py, which mimics the 'cam' tool. Four rendering backends are\n> added:\n> \n> * null - Do nothing\n> * kms - Use KMS with dmabufs\n> * qt - SW render on a Qt window\n> * qtgl - OpenGL render on a Qt window\n> \n> All the renderers handle only a few pixel formats, and especially the GL\n> renderer is just a prototype.\n> \n> Signed-off-by: Tomi Valkeinen \n> ---\n> src/py/test/cam.py | 464 ++++++++++++++++++++++++++++++++++++++\n> src/py/test/cam_kms.py | 185 +++++++++++++++\n> src/py/test/cam_null.py | 46 ++++\n> src/py/test/cam_qt.py | 355 +++++++++++++++++++++++++++++\n> src/py/test/cam_qtgl.py | 385 +++++++++++++++++++++++++++++++\n> src/py/test/gl_helpers.py | 67 ++++++\n\nWhile these are 'test' apps, they're not quite 'tests'.\n\nI think these should live under src/py/pycam? Or some such name?\nEssentially they will likely form the basis of 'example python usage'...\n\nI'd quite like to see something (anything?) that counts as a unittest\nintegrated into /test/py/ that somehow validates the APIs that are\ncreated.\n\nEven if it's only basic for now to construct a camera, or something, so\nthat when built/enabled, the python code can be linked into the test\nframework with 'ninja -C build test'.\n\nI wonder if a top-level TODO file under src/py/TODO and/or\nsrc/py/{test/pycam}/TODO will help clearly mark things that are known\nnot to be implemented ?\n\n\nBeyond that, I think this series would benefit from early integration so\nit can be more thoroughly used and devloped rather than trying to\n'perfect' it out in isolation.\n\n--\nKieran\n\n\n> 6 files changed, 1502 insertions(+)\n> create mode 100755 src/py/test/cam.py\n> create mode 100644 src/py/test/cam_kms.py\n> create mode 100644 src/py/test/cam_null.py\n> create mode 100644 src/py/test/cam_qt.py\n> create mode 100644 src/py/test/cam_qtgl.py\n> create mode 100644 src/py/test/gl_helpers.py\n> \n> diff --git a/src/py/test/cam.py b/src/py/test/cam.py\n> new file mode 100755\n> index 00000000..48df01cf\n> --- /dev/null\n> +++ b/src/py/test/cam.py\n> @@ -0,0 +1,464 @@\n> +#!/usr/bin/python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +\n> +import pycamera as pycam\n> +import time\n> +import binascii\n> +import argparse\n> +import selectors\n> +import os\n> +import sys\n> +\n> +class CustomCameraAction(argparse.Action):\n> + def __call__(self, parser, namespace, values, option_string=None):\n> + print(self.dest, values)\n> +\n> + if not \"camera\" in namespace or namespace.camera == None:\n> + setattr(namespace, \"camera\", [])\n> +\n> + previous = namespace.camera\n> + previous.append((self.dest, values))\n> + setattr(namespace, \"camera\", previous)\n> +\n> +class CustomAction(argparse.Action):\n> + def __init__(self, option_strings, dest, **kwargs):\n> + super().__init__(option_strings, dest, default={}, **kwargs)\n> +\n> + def __call__(self, parser, namespace, values, option_string=None):\n> + if len(namespace.camera) == 0:\n> + print(f\"Option {option_string} requires a --camera context\")\n> + sys.exit(-1)\n> +\n> + if self.type == bool:\n> + values = True\n> +\n> + current = namespace.camera[-1]\n> +\n> + data = getattr(namespace, self.dest)\n> +\n> + if self.nargs == \"+\":\n> + if not current in data:\n> + data[current] = []\n> +\n> + data[current] += values\n> + else:\n> + data[current] = values\n> +\n> +\n> +\n> +def do_cmd_list(cm):\n> + print(\"Available cameras:\")\n> +\n> + for idx,c in enumerate(cm.cameras):\n> + print(f\"{idx + 1}: {c.id}\")\n> +\n> +def do_cmd_list_props(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + print(\"Properties for\", ctx[\"id\"])\n> +\n> + for name, prop in camera.properties.items():\n> + print(\"\\t{}: {}\".format(name, prop))\n> +\n> +def do_cmd_list_controls(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + print(\"Controls for\", ctx[\"id\"])\n> +\n> + for name, prop in camera.controls.items():\n> + print(\"\\t{}: {}\".format(name, prop))\n> +\n> +def do_cmd_info(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + print(\"Stream info for\", ctx[\"id\"])\n> +\n> + roles = [pycam.StreamRole.Viewfinder]\n> +\n> + camconfig = camera.generateConfiguration(roles)\n> + if camconfig == None:\n> + raise Exception(\"Generating config failed\")\n> +\n> + for i, stream_config in enumerate(camconfig):\n> + print(\"\\t{}: {}\".format(i, stream_config.toString()))\n> +\n> + formats = stream_config.formats\n> + for fmt in formats.pixelFormats:\n> + print(\"\\t * Pixelformat:\", fmt, formats.range(fmt))\n> +\n> + for size in formats.sizes(fmt):\n> + print(\"\\t -\", size)\n> +\n> +def acquire(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + camera.acquire()\n> +\n> +def release(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + camera.release()\n> +\n> +def parse_streams(ctx):\n> + streams = []\n> +\n> + for stream_desc in ctx[\"opt-stream\"]:\n> + stream_opts = {\"role\": pycam.StreamRole.Viewfinder}\n> +\n> + for stream_opt in stream_desc.split(\",\"):\n> + if stream_opt == 0:\n> + continue\n> +\n> + arr = stream_opt.split(\"=\")\n> + if len(arr) != 2:\n> + print(\"Bad stream option\", stream_opt)\n> + sys.exit(-1)\n> +\n> + key = arr[0]\n> + value = arr[1]\n> +\n> + if key in [\"width\", \"height\"]:\n> + value = int(value)\n> + elif key == \"role\":\n> + rolemap = {\n> + \"still\": pycam.StreamRole.StillCapture,\n> + \"raw\": pycam.StreamRole.Raw,\n> + \"video\": pycam.StreamRole.VideoRecording,\n> + \"viewfinder\": pycam.StreamRole.Viewfinder,\n> + }\n> +\n> + role = rolemap.get(value.lower(), None)\n> +\n> + if role == None:\n> + print(\"Bad stream role\", value)\n> + sys.exit(-1)\n> +\n> + value = role\n> + elif key == \"pixelformat\":\n> + pass\n> + else:\n> + print(\"Bad stream option key\", key)\n> + sys.exit(-1)\n> +\n> + stream_opts[key] = value\n> +\n> + streams.append(stream_opts)\n> +\n> + return streams\n> +\n> +def configure(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + streams = parse_streams(ctx)\n> +\n> + roles = [opts[\"role\"] for opts in streams]\n> +\n> + camconfig = camera.generateConfiguration(roles)\n> + if camconfig == None:\n> + raise Exception(\"Generating config failed\")\n> +\n> + for idx,stream_opts in enumerate(streams):\n> + stream_config = camconfig.at(idx)\n> +\n> + if \"width\" in stream_opts and \"height\" in stream_opts:\n> + stream_config.size = (stream_opts[\"width\"], stream_opts[\"height\"])\n> +\n> + if \"pixelformat\" in stream_opts:\n> + stream_config.fmt = stream_opts[\"pixelformat\"]\n> +\n> + stat = camconfig.validate()\n> +\n> + if stat == pycam.ConfigurationStatus.Invalid:\n> + print(\"Camera configuration invalid\")\n> + exit(-1)\n> + elif stat == pycam.ConfigurationStatus.Adjusted:\n> + if ctx[\"opt-strict-formats\"]:\n> + print(\"Adjusting camera configuration disallowed by --strict-formats argument\")\n> + exit(-1)\n> +\n> + print(\"Camera configuration adjusted\")\n> +\n> + r = camera.configure(camconfig);\n> + if r != 0:\n> + raise Exception(\"Configure failed\")\n> +\n> + ctx[\"stream-names\"] = {}\n> + ctx[\"streams\"] = []\n> +\n> + for idx, stream_config in enumerate(camconfig):\n> + stream = stream_config.stream\n> + ctx[\"streams\"].append(stream)\n> + ctx[\"stream-names\"][stream] = \"stream\" + str(idx)\n> + print(\"{}-{}: stream config {}\".format(ctx[\"id\"], ctx[\"stream-names\"][stream], stream.configuration.toString()))\n> +\n> +def alloc_buffers(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + allocator = pycam.FrameBufferAllocator(camera);\n> +\n> + for idx, stream in enumerate(ctx[\"streams\"]):\n> + ret = allocator.allocate(stream)\n> + if ret < 0:\n> + print(\"Can't allocate buffers\")\n> + exit(-1)\n> +\n> + allocated = len(allocator.buffers(stream))\n> +\n> + print(\"{}-{}: Allocated {} buffers\".format(ctx[\"id\"], ctx[\"stream-names\"][stream], allocated))\n> +\n> + ctx[\"allocator\"] = allocator\n> +\n> +def create_requests(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + ctx[\"requests\"] = []\n> +\n> + # Identify the stream with the least number of buffers\n> + num_bufs = min([len(ctx[\"allocator\"].buffers(stream)) for stream in ctx[\"streams\"]])\n> +\n> + requests = []\n> +\n> + for buf_num in range(num_bufs):\n> + request = camera.createRequest()\n> +\n> + if request == None:\n> + print(\"Can't create request\")\n> + exit(-1)\n> +\n> + for stream in ctx[\"streams\"]:\n> + buffers = ctx[\"allocator\"].buffers(stream)\n> + buffer = buffers[buf_num]\n> +\n> + ret = request.addBuffer(stream, buffer)\n> + if ret < 0:\n> + print(\"Can't set buffer for request\")\n> + exit(-1)\n> +\n> + requests.append(request)\n> +\n> + ctx[\"requests\"] = requests\n> +\n> +def start(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + camera.start()\n> +\n> +def stop(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + camera.stop()\n> +\n> +def queue_requests(ctx):\n> + camera = ctx[\"camera\"]\n> +\n> + for request in ctx[\"requests\"]:\n> + camera.queueRequest(request)\n> + ctx[\"reqs-queued\"] += 1\n> +\n> + del ctx[\"requests\"]\n> +\n> +def capture_init(contexts):\n> + for ctx in contexts:\n> + acquire(ctx)\n> +\n> + for ctx in contexts:\n> + configure(ctx)\n> +\n> + for ctx in contexts:\n> + alloc_buffers(ctx)\n> +\n> + for ctx in contexts:\n> + create_requests(ctx)\n> +\n> +def capture_start(contexts):\n> + for ctx in contexts:\n> + start(ctx)\n> +\n> + for ctx in contexts:\n> + queue_requests(ctx)\n> +\n> +# Called from renderer when there is a libcamera event\n> +def event_handler(state):\n> + cm = state[\"cm\"]\n> + contexts = state[\"contexts\"]\n> +\n> + data = os.read(cm.efd, 8)\n> +\n> + reqs = cm.getReadyRequests()\n> +\n> + for req in reqs:\n> + ctx = next(ctx for ctx in contexts if ctx[\"camera\"] == req.camera)\n> + request_handler(state, ctx, req)\n> +\n> + running = any(ctx[\"reqs-completed\"] < ctx[\"opt-capture\"] for ctx in contexts)\n> + return running\n> +\n> +def request_handler(state, ctx, req):\n> + camera = ctx[\"camera\"]\n> +\n> + if req.status != pycam.RequestStatus.Complete:\n> + raise Exception(\"{}: Request failed: {}\".format(ctx[\"id\"], req.status))\n> +\n> + buffers = req.buffers\n> +\n> + # Compute the frame rate. The timestamp is arbitrarily retrieved from\n> + # the first buffer, as all buffers should have matching timestamps.\n> + ts = buffers[next(iter(buffers))].metadata.timestamp\n> + last = ctx.get(\"last\", 0)\n> + fps = 1000000000.0 / (ts - last) if (last != 0 and (ts - last) != 0) else 0\n> + ctx[\"last\"] = ts\n> + ctx[\"fps\"] = fps\n> +\n> + for stream, fb in buffers.items():\n> + stream_name = ctx[\"stream-names\"][stream]\n> +\n> + crcs = []\n> + if ctx[\"opt-crc\"]:\n> + with fb.mmap(0) as b:\n> + crc = binascii.crc32(b)\n> + crcs.append(crc)\n> +\n> + meta = fb.metadata\n> +\n> + print(\"{:.6f} ({:.2f} fps) {}-{}: seq {}, bytes {}, CRCs {}\"\n> + .format(ts / 1000000000, fps,\n> + ctx[\"id\"], stream_name,\n> + meta.sequence, meta.bytesused,\n> + crcs))\n> +\n> + if ctx[\"opt-metadata\"]:\n> + reqmeta = req.metadata\n> + for ctrl, val in reqmeta.items():\n> + print(f\"\\t{ctrl} = {val}\")\n> +\n> + if ctx[\"opt-save-frames\"]:\n> + with fb.mmap(0) as b:\n> + filename = \"frame-{}-{}-{}.data\".format(ctx[\"id\"], stream_name, ctx[\"reqs-completed\"])\n> + with open(filename, \"wb\") as f:\n> + f.write(b)\n> +\n> + state[\"renderer\"].request_handler(ctx, req);\n> +\n> + ctx[\"reqs-completed\"] += 1\n> +\n> +# Called from renderer when it has finished with a request\n> +def request_prcessed(ctx, req):\n> + camera = ctx[\"camera\"]\n> +\n> + if ctx[\"reqs-queued\"] < ctx[\"opt-capture\"]:\n> + req.reuse()\n> + camera.queueRequest(req)\n> + ctx[\"reqs-queued\"] += 1\n> +\n> +def capture_deinit(contexts):\n> + for ctx in contexts:\n> + stop(ctx)\n> +\n> + for ctx in contexts:\n> + release(ctx)\n> +\n> +def do_cmd_capture(state):\n> + capture_init(state[\"contexts\"])\n> +\n> + renderer = state[\"renderer\"]\n> +\n> + renderer.setup()\n> +\n> + capture_start(state[\"contexts\"])\n> +\n> + renderer.run()\n> +\n> + capture_deinit(state[\"contexts\"])\n> +\n> +def main():\n> + parser = argparse.ArgumentParser()\n> + # global options\n> + parser.add_argument(\"-l\", \"--list\", action=\"store_true\", help=\"List all cameras\")\n> + parser.add_argument(\"-c\", \"--camera\", type=int, action=\"extend\", nargs=1, default=[], help=\"Specify which camera to operate on, by index\")\n> + parser.add_argument(\"-p\", \"--list-properties\", action=\"store_true\", help=\"List cameras properties\")\n> + parser.add_argument(\"--list-controls\", action=\"store_true\", help=\"List cameras controls\")\n> + parser.add_argument(\"-I\", \"--info\", action=\"store_true\", help=\"Display information about stream(s)\")\n> + parser.add_argument(\"-R\", \"--renderer\", default=\"null\", help=\"Renderer (null, kms, qt, qtgl)\")\n> +\n> + # per camera options\n> + parser.add_argument(\"-C\", \"--capture\", nargs=\"?\", type=int, const=1000000, action=CustomAction, help=\"Capture until interrupted by user or until CAPTURE frames captured\")\n> + parser.add_argument(\"--crc\", nargs=0, type=bool, action=CustomAction, help=\"Print CRC32 for captured frames\")\n> + parser.add_argument(\"--save-frames\", nargs=0, type=bool, action=CustomAction, help=\"Save captured frames to files\")\n> + parser.add_argument(\"--metadata\", nargs=0, type=bool, action=CustomAction, help=\"Print the metadata for completed requests\")\n> + parser.add_argument(\"--strict-formats\", type=bool, nargs=0, action=CustomAction, help=\"Do not allow requested stream format(s) to be adjusted\")\n> + parser.add_argument(\"-s\", \"--stream\", nargs=\"+\", action=CustomAction)\n> + args = parser.parse_args()\n> +\n> + cm = pycam.CameraManager.singleton()\n> +\n> + if args.list:\n> + do_cmd_list(cm)\n> +\n> + contexts = []\n> +\n> + for cam_idx in args.camera:\n> + camera = next((c for i,c in enumerate(cm.cameras) if i + 1 == cam_idx), None)\n> +\n> + if camera == None:\n> + print(\"Unable to find camera\", cam_idx)\n> + return -1\n> +\n> + contexts.append({\n> + \"camera\": camera,\n> + \"idx\": cam_idx,\n> + \"id\": \"cam\" + str(cam_idx),\n> + \"reqs-queued\": 0,\n> + \"reqs-completed\": 0,\n> + \"opt-capture\": args.capture.get(cam_idx, False),\n> + \"opt-crc\": args.crc.get(cam_idx, False),\n> + \"opt-save-frames\": args.save_frames.get(cam_idx, False),\n> + \"opt-metadata\": args.metadata.get(cam_idx, False),\n> + \"opt-strict-formats\": args.strict_formats.get(cam_idx, False),\n> + \"opt-stream\": args.stream.get(cam_idx, [\"role=viewfinder\"]),\n> + })\n> +\n> + for ctx in contexts:\n> + print(\"Using camera {} as {}\".format(ctx[\"camera\"].id, ctx[\"id\"]))\n> +\n> + for ctx in contexts:\n> + if args.list_properties:\n> + do_cmd_list_props(ctx)\n> + if args.list_controls:\n> + do_cmd_list_controls(ctx)\n> + if args.info:\n> + do_cmd_info(ctx)\n> +\n> + if args.capture:\n> +\n> + state = {\n> + \"cm\": cm,\n> + \"contexts\": contexts,\n> + \"event_handler\": event_handler,\n> + \"request_prcessed\": request_prcessed,\n> + }\n> +\n> + if args.renderer == \"null\":\n> + import cam_null\n> + renderer = cam_null.NullRenderer(state)\n> + elif args.renderer == \"kms\":\n> + import cam_kms\n> + renderer = cam_kms.KMSRenderer(state)\n> + elif args.renderer == \"qt\":\n> + import cam_qt\n> + renderer = cam_qt.QtRenderer(state)\n> + elif args.renderer == \"qtgl\":\n> + import cam_qtgl\n> + renderer = cam_qtgl.QtRenderer(state)\n> + else:\n> + print(\"Bad renderer\", args.renderer)\n> + return -1\n> +\n> + state[\"renderer\"] = renderer\n> +\n> + do_cmd_capture(state)\n> +\n> + return 0\n> +\n> +if __name__ == \"__main__\":\n> + sys.exit(main())\n> diff --git a/src/py/test/cam_kms.py b/src/py/test/cam_kms.py\n> new file mode 100644\n> index 00000000..fb0e6375\n> --- /dev/null\n> +++ b/src/py/test/cam_kms.py\n> @@ -0,0 +1,185 @@\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +\n> +import sys\n> +import selectors\n> +import pykms\n> +\n> +FMT_MAP = {\n> + \"RGB888\": pykms.PixelFormat.RGB888,\n> + \"YUYV\": pykms.PixelFormat.YUYV,\n> + \"ARGB8888\": pykms.PixelFormat.ARGB8888,\n> + \"XRGB8888\": pykms.PixelFormat.XRGB8888,\n> +}\n> +\n> +class KMSRenderer:\n> + def __init__(self, state):\n> + self.state = state\n> +\n> + self.cm = state[\"cm\"]\n> + self.contexts = state[\"contexts\"]\n> + self.running = False\n> +\n> + card = pykms.Card()\n> +\n> + res = pykms.ResourceManager(card)\n> + conn = res.reserve_connector()\n> + crtc = res.reserve_crtc(conn)\n> + mode = conn.get_default_mode()\n> + modeb = mode.to_blob(card)\n> +\n> + req = pykms.AtomicReq(card)\n> + req.add_connector(conn, crtc)\n> + req.add_crtc(crtc, modeb)\n> + r = req.commit_sync(allow_modeset = True)\n> + assert(r == 0)\n> +\n> + self.card = card\n> + self.resman = res\n> + self.crtc = crtc\n> + self.mode = mode\n> +\n> + self.bufqueue = []\n> + self.current = None\n> + self.next = None\n> + self.cam_2_drm = {}\n> +\n> + # KMS\n> +\n> + def close(self):\n> + req = pykms.AtomicReq(self.card)\n> + for s in self.streams:\n> + req.add_plane(s[\"plane\"], None, None, dst=(0, 0, 0, 0))\n> + req.commit()\n> +\n> + def add_plane(self, req, stream, fb):\n> + s = next(s for s in self.streams if s[\"stream\"] == stream)\n> + idx = s[\"idx\"]\n> + plane = s[\"plane\"]\n> +\n> + if idx % 2 == 0:\n> + x = 0\n> + else:\n> + x = self.mode.hdisplay - fb.width\n> +\n> + if idx // 2 == 0:\n> + y = 0\n> + else:\n> + y = self.mode.vdisplay - fb.height\n> +\n> + req.add_plane(plane, fb, self.crtc, dst=(x, y, fb.width, fb.height))\n> +\n> + def apply_request(self, drmreq):\n> +\n> + buffers = drmreq[\"camreq\"].buffers\n> +\n> + for stream, fb in buffers.items():\n> + drmfb = self.cam_2_drm.get(fb, None)\n> +\n> + req = pykms.AtomicReq(self.card)\n> + self.add_plane(req, stream, drmfb)\n> + req.commit()\n> +\n> + def handle_page_flip(self, frame, time):\n> + old = self.current\n> + self.current = self.next\n> +\n> + if len(self.bufqueue) > 0:\n> + self.next = self.bufqueue.pop(0)\n> + else:\n> + self.next = None\n> +\n> + if self.next:\n> + drmreq = self.next\n> +\n> + self.apply_request(drmreq)\n> +\n> + if old:\n> + req = old[\"camreq\"]\n> + ctx = old[\"camctx\"]\n> + self.state[\"request_prcessed\"](ctx, req)\n> +\n> + def queue(self, drmreq):\n> + if not self.next:\n> + self.next = drmreq\n> + self.apply_request(drmreq)\n> + else:\n> + self.bufqueue.append(drmreq)\n> +\n> + # libcamera\n> +\n> + def setup(self):\n> + self.streams = []\n> +\n> + idx = 0\n> + for ctx in self.contexts:\n> + camera = ctx[\"camera\"]\n> +\n> + for stream in ctx[\"streams\"]:\n> +\n> + cfg = stream.configuration\n> + fmt = cfg.fmt\n> + fmt = FMT_MAP[fmt]\n> +\n> + plane = self.resman.reserve_generic_plane(self.crtc, fmt)\n> + assert(plane != None)\n> +\n> + self.streams.append({\n> + \"idx\": idx,\n> + \"stream\": stream,\n> + \"plane\": plane,\n> + \"fmt\": fmt,\n> + \"size\": cfg.size,\n> + })\n> +\n> + for fb in ctx[\"allocator\"].buffers(stream):\n> + w, h = cfg.size\n> + stride = cfg.stride\n> + fd = fb.fd(0)\n> + drmfb = pykms.DmabufFramebuffer(self.card, w, h, fmt,\n> + [fd], [stride], [0])\n> + self.cam_2_drm[fb] = drmfb\n> +\n> + idx += 1\n> +\n> +\n> + def readdrm(self, fileobj):\n> + for ev in self.card.read_events():\n> + if ev.type == pykms.DrmEventType.FLIP_COMPLETE:\n> + self.handle_page_flip(ev.seq, ev.time)\n> +\n> + def readcam(self, fd):\n> + self.running = self.state[\"event_handler\"](self.state)\n> +\n> + def readkey(self, fileobj):\n> + sys.stdin.readline()\n> + self.running = False\n> +\n> + def run(self):\n> + print(\"Capturing...\")\n> +\n> + self.running = True\n> +\n> + sel = selectors.DefaultSelector()\n> + sel.register(self.card.fd, selectors.EVENT_READ, self.readdrm)\n> + sel.register(self.cm.efd, selectors.EVENT_READ, self.readcam)\n> + sel.register(sys.stdin, selectors.EVENT_READ, self.readkey)\n> +\n> + print(\"Press enter to exit\")\n> +\n> + while self.running:\n> + events = sel.select()\n> + for key, mask in events:\n> + callback = key.data\n> + callback(key.fileobj)\n> +\n> + print(\"Exiting...\")\n> +\n> + def request_handler(self, ctx, req):\n> +\n> + drmreq = {\n> + \"camctx\": ctx,\n> + \"camreq\": req,\n> + }\n> +\n> + self.queue(drmreq)\n> diff --git a/src/py/test/cam_null.py b/src/py/test/cam_null.py\n> new file mode 100644\n> index 00000000..3935f5b6\n> --- /dev/null\n> +++ b/src/py/test/cam_null.py\n> @@ -0,0 +1,46 @@\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +\n> +import sys\n> +import selectors\n> +\n> +class NullRenderer:\n> + def __init__(self, state):\n> + self.state = state\n> +\n> + self.cm = state[\"cm\"]\n> + self.contexts = state[\"contexts\"]\n> +\n> + self.running = False\n> +\n> + def setup(self):\n> + pass\n> +\n> + def run(self):\n> + print(\"Capturing...\")\n> +\n> + self.running = True\n> +\n> + sel = selectors.DefaultSelector()\n> + sel.register(self.cm.efd, selectors.EVENT_READ, self.readcam)\n> + sel.register(sys.stdin, selectors.EVENT_READ, self.readkey)\n> +\n> + print(\"Press enter to exit\")\n> +\n> + while self.running:\n> + events = sel.select()\n> + for key, mask in events:\n> + callback = key.data\n> + callback(key.fileobj)\n> +\n> + print(\"Exiting...\")\n> +\n> + def readcam(self, fd):\n> + self.running = self.state[\"event_handler\"](self.state)\n> +\n> + def readkey(self, fileobj):\n> + sys.stdin.readline()\n> + self.running = False\n> +\n> + def request_handler(self, ctx, req):\n> + self.state[\"request_prcessed\"](ctx, req)\n> diff --git a/src/py/test/cam_qt.py b/src/py/test/cam_qt.py\n> new file mode 100644\n> index 00000000..3ff12df6\n> --- /dev/null\n> +++ b/src/py/test/cam_qt.py\n> @@ -0,0 +1,355 @@\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +#\n> +# Debayering code from PiCamera documentation\n> +\n> +from PyQt5 import QtCore, QtGui, QtWidgets\n> +from io import BytesIO\n> +from PIL import Image\n> +from PIL.ImageQt import ImageQt\n> +import numpy as np\n> +from numpy.lib.stride_tricks import as_strided\n> +import sys\n> +\n> +def rgb_to_pix(rgb):\n> + img = Image.frombuffer(\"RGB\", (rgb.shape[1], rgb.shape[0]), rgb)\n> + qim = ImageQt(img).copy()\n> + pix = QtGui.QPixmap.fromImage(qim)\n> + return pix\n> +\n> +\n> +def separate_components(data, r0, g0, g1, b0):\n> + # Now to split the data up into its red, green, and blue components. The\n> + # Bayer pattern of the OV5647 sensor is BGGR. In other words the first\n> + # row contains alternating green/blue elements, the second row contains\n> + # alternating red/green elements, and so on as illustrated below:\n> + #\n> + # GBGBGBGBGBGBGB\n> + # RGRGRGRGRGRGRG\n> + # GBGBGBGBGBGBGB\n> + # RGRGRGRGRGRGRG\n> + #\n> + # Please note that if you use vflip or hflip to change the orientation\n> + # of the capture, you must flip the Bayer pattern accordingly\n> +\n> + rgb = np.zeros(data.shape + (3,), dtype=data.dtype)\n> + rgb[r0[1]::2, r0[0]::2, 0] = data[r0[1]::2, r0[0]::2] # Red\n> + rgb[g0[1]::2, g0[0]::2, 1] = data[g0[1]::2, g0[0]::2] # Green\n> + rgb[g1[1]::2, g1[0]::2, 1] = data[g1[1]::2, g1[0]::2] # Green\n> + rgb[b0[1]::2, b0[0]::2, 2] = data[b0[1]::2, b0[0]::2] # Blue\n> +\n> + return rgb\n> +\n> +def demosaic(rgb, r0, g0, g1, b0):\n> + # At this point we now have the raw Bayer data with the correct values\n> + # and colors but the data still requires de-mosaicing and\n> + # post-processing. If you wish to do this yourself, end the script here!\n> + #\n> + # Below we present a fairly naive de-mosaic method that simply\n> + # calculates the weighted average of a pixel based on the pixels\n> + # surrounding it. The weighting is provided b0[1] a b0[1]te representation of\n> + # the Bayer filter which we construct first:\n> +\n> + bayer = np.zeros(rgb.shape, dtype=np.uint8)\n> + bayer[r0[1]::2, r0[0]::2, 0] = 1 # Red\n> + bayer[g0[1]::2, g0[0]::2, 1] = 1 # Green\n> + bayer[g1[1]::2, g1[0]::2, 1] = 1 # Green\n> + bayer[b0[1]::2, b0[0]::2, 2] = 1 # Blue\n> +\n> + # Allocate an array to hold our output with the same shape as the input\n> + # data. After this we define the size of window that will be used to\n> + # calculate each weighted average (3x3). Then we pad out the rgb and\n> + # bayer arrays, adding blank pixels at their edges to compensate for the\n> + # size of the window when calculating averages for edge pixels.\n> +\n> + output = np.empty(rgb.shape, dtype=rgb.dtype)\n> + window = (3, 3)\n> + borders = (window[0] - 1, window[1] - 1)\n> + border = (borders[0] // 2, borders[1] // 2)\n> +\n> + #rgb_pad = np.zeros((\n> + # rgb.shape[0] + borders[0],\n> + # rgb.shape[1] + borders[1],\n> + # rgb.shape[2]), dtype=rgb.dtype)\n> + #rgb_pad[\n> + # border[0]:rgb_pad.shape[0] - border[0],\n> + # border[1]:rgb_pad.shape[1] - border[1],\n> + # :] = rgb\n> + #rgb = rgb_pad\n> + #\n> + #bayer_pad = np.zeros((\n> + # bayer.shape[0] + borders[0],\n> + # bayer.shape[1] + borders[1],\n> + # bayer.shape[2]), dtype=bayer.dtype)\n> + #bayer_pad[\n> + # border[0]:bayer_pad.shape[0] - border[0],\n> + # border[1]:bayer_pad.shape[1] - border[1],\n> + # :] = bayer\n> + #bayer = bayer_pad\n> +\n> + # In numpy >=1.7.0 just use np.pad (version in Raspbian is 1.6.2 at the\n> + # time of writing...)\n> + #\n> + rgb = np.pad(rgb, [\n> + (border[0], border[0]),\n> + (border[1], border[1]),\n> + (0, 0),\n> + ], 'constant')\n> + bayer = np.pad(bayer, [\n> + (border[0], border[0]),\n> + (border[1], border[1]),\n> + (0, 0),\n> + ], 'constant')\n> +\n> + # For each plane in the RGB data, we use a nifty numpy trick\n> + # (as_strided) to construct a view over the plane of 3x3 matrices. We do\n> + # the same for the bayer array, then use Einstein summation on each\n> + # (np.sum is simpler, but copies the data so it's slower), and divide\n> + # the results to get our weighted average:\n> +\n> + for plane in range(3):\n> + p = rgb[..., plane]\n> + b = bayer[..., plane]\n> + pview = as_strided(p, shape=(\n> + p.shape[0] - borders[0],\n> + p.shape[1] - borders[1]) + window, strides=p.strides * 2)\n> + bview = as_strided(b, shape=(\n> + b.shape[0] - borders[0],\n> + b.shape[1] - borders[1]) + window, strides=b.strides * 2)\n> + psum = np.einsum('ijkl->ij', pview)\n> + bsum = np.einsum('ijkl->ij', bview)\n> + output[..., plane] = psum // bsum\n> +\n> + return output\n> +\n> +\n> +\n> +\n> +def to_rgb(fmt, size, data):\n> + w = size[0]\n> + h = size[1]\n> +\n> + if fmt == \"YUYV\":\n> + # YUV422\n> + yuyv = data.reshape((h, w // 2 * 4))\n> +\n> + # YUV444\n> + yuv = np.empty((h, w, 3), dtype=np.uint8)\n> + yuv[:, :, 0] = yuyv[:, 0::2] # Y\n> + yuv[:, :, 1] = yuyv[:, 1::4].repeat(2, axis=1) # U\n> + yuv[:, :, 2] = yuyv[:, 3::4].repeat(2, axis=1) # V\n> +\n> + m = np.array([\n> + [ 1.0, 1.0, 1.0],\n> + [-0.000007154783816076815, -0.3441331386566162, 1.7720025777816772],\n> + [ 1.4019975662231445, -0.7141380310058594 , 0.00001542569043522235]\n> + ])\n> +\n> + rgb = np.dot(yuv, m)\n> + rgb[:, :, 0] -= 179.45477266423404\n> + rgb[:, :, 1] += 135.45870971679688\n> + rgb[:, :, 2] -= 226.8183044444304\n> + rgb = rgb.astype(np.uint8)\n> +\n> + elif fmt == \"RGB888\":\n> + rgb = data.reshape((h, w, 3))\n> + rgb[:, :, [0, 1, 2]] = rgb[:, :, [2, 1, 0]]\n> +\n> + elif fmt == \"BGR888\":\n> + rgb = data.reshape((h, w, 3))\n> +\n> + elif fmt in [\"ARGB8888\", \"XRGB8888\"]:\n> + rgb = data.reshape((h, w, 4))\n> + rgb = np.flip(rgb, axis=2)\n> + # drop alpha component\n> + rgb = np.delete(rgb, np.s_[0::4], axis=2)\n> +\n> + elif fmt.startswith(\"S\"):\n> + bayer_pattern = fmt[1:5]\n> + bitspp = int(fmt[5:])\n> +\n> + # TODO: shifting leaves the lowest bits 0\n> + if bitspp == 8:\n> + data = data.reshape((h, w))\n> + data = data.astype(np.uint16) << 8\n> + elif bitspp in [10, 12]:\n> + data = data.view(np.uint16)\n> + data = data.reshape((h, w))\n> + data = data << (16 - bitspp)\n> + else:\n> + raise Exception(\"Bad bitspp:\" + str(bitspp))\n> +\n> + idx = bayer_pattern.find(\"R\")\n> + assert(idx != -1)\n> + r0 = (idx % 2, idx // 2)\n> +\n> + idx = bayer_pattern.find(\"G\")\n> + assert(idx != -1)\n> + g0 = (idx % 2, idx // 2)\n> +\n> + idx = bayer_pattern.find(\"G\", idx + 1)\n> + assert(idx != -1)\n> + g1 = (idx % 2, idx // 2)\n> +\n> + idx = bayer_pattern.find(\"B\")\n> + assert(idx != -1)\n> + b0 = (idx % 2, idx // 2)\n> +\n> + rgb = separate_components(data, r0, g0, g1, b0)\n> + rgb = demosaic(rgb, r0, g0, g1, b0)\n> + rgb = (rgb >> 8).astype(np.uint8)\n> +\n> + else:\n> + rgb = None\n> +\n> + return rgb\n> +\n> +\n> +class QtRenderer:\n> + def __init__(self, state):\n> + self.state = state\n> +\n> + self.cm = state[\"cm\"]\n> + self.contexts = state[\"contexts\"]\n> +\n> + def setup(self):\n> + self.app = QtWidgets.QApplication([])\n> +\n> + windows = []\n> +\n> + for ctx in self.contexts:\n> + camera = ctx[\"camera\"]\n> +\n> + for stream in ctx[\"streams\"]:\n> + fmt = stream.configuration.fmt\n> + size = stream.configuration.size\n> +\n> + window = MainWindow(ctx, stream)\n> + window.setAttribute(QtCore.Qt.WA_ShowWithoutActivating)\n> + window.show()\n> + windows.append(window)\n> +\n> + self.windows = windows\n> +\n> + def run(self):\n> + camnotif = QtCore.QSocketNotifier(self.cm.efd, QtCore.QSocketNotifier.Read)\n> + camnotif.activated.connect(lambda x: self.readcam())\n> +\n> + keynotif = QtCore.QSocketNotifier(sys.stdin.fileno(), QtCore.QSocketNotifier.Read)\n> + keynotif.activated.connect(lambda x: self.readkey())\n> +\n> + print(\"Capturing...\")\n> +\n> + self.app.exec()\n> +\n> + print(\"Exiting...\")\n> +\n> + def readcam(self):\n> + running = self.state[\"event_handler\"](self.state)\n> +\n> + if not running:\n> + self.app.quit()\n> +\n> + def readkey(self):\n> + sys.stdin.readline()\n> + self.app.quit()\n> +\n> + def request_handler(self, ctx, req):\n> + buffers = req.buffers\n> +\n> + for stream, fb in buffers.items():\n> + wnd = next(wnd for wnd in self.windows if wnd.stream == stream)\n> +\n> + wnd.handle_request(stream, fb)\n> +\n> + self.state[\"request_prcessed\"](ctx, req)\n> +\n> + def cleanup(self):\n> + for w in self.windows:\n> + w.close()\n> +\n> +\n> +class MainWindow(QtWidgets.QWidget):\n> + def __init__(self, ctx, stream):\n> + super().__init__()\n> +\n> + self.ctx = ctx\n> + self.stream = stream\n> +\n> + self.label = QtWidgets.QLabel()\n> +\n> + windowLayout = QtWidgets.QHBoxLayout()\n> + self.setLayout(windowLayout)\n> +\n> + windowLayout.addWidget(self.label)\n> +\n> + controlsLayout = QtWidgets.QVBoxLayout()\n> + windowLayout.addLayout(controlsLayout)\n> +\n> + windowLayout.addStretch()\n> +\n> + group = QtWidgets.QGroupBox(\"Info\")\n> + groupLayout = QtWidgets.QVBoxLayout()\n> + group.setLayout(groupLayout)\n> + controlsLayout.addWidget(group)\n> +\n> + lab = QtWidgets.QLabel(ctx[\"id\"])\n> + groupLayout.addWidget(lab)\n> +\n> + self.frameLabel = QtWidgets.QLabel()\n> + groupLayout.addWidget(self.frameLabel)\n> +\n> +\n> + group = QtWidgets.QGroupBox(\"Properties\")\n> + groupLayout = QtWidgets.QVBoxLayout()\n> + group.setLayout(groupLayout)\n> + controlsLayout.addWidget(group)\n> +\n> + camera = ctx[\"camera\"]\n> +\n> + for k, v in camera.properties.items():\n> + lab = QtWidgets.QLabel()\n> + lab.setText(k + \" = \" + str(v))\n> + groupLayout.addWidget(lab)\n> +\n> + group = QtWidgets.QGroupBox(\"Controls\")\n> + groupLayout = QtWidgets.QVBoxLayout()\n> + group.setLayout(groupLayout)\n> + controlsLayout.addWidget(group)\n> +\n> + for k, (min, max, default) in camera.controls.items():\n> + lab = QtWidgets.QLabel()\n> + lab.setText(\"{} = {}/{}/{}\".format(k, min, max, default))\n> + groupLayout.addWidget(lab)\n> +\n> + controlsLayout.addStretch()\n> +\n> + def buf_to_qpixmap(self, stream, fb):\n> + with fb.mmap(0) as b:\n> + cfg = stream.configuration\n> + w, h = cfg.size\n> + pitch = cfg.stride\n> +\n> + if cfg.fmt == \"MJPEG\":\n> + img = Image.open(BytesIO(b))\n> + qim = ImageQt(img).copy()\n> + pix = QtGui.QPixmap.fromImage(qim)\n> + else:\n> + data = np.array(b, dtype=np.uint8)\n> + rgb = to_rgb(cfg.fmt, cfg.size, data)\n> +\n> + if rgb is None:\n> + raise Exception(\"Format not supported: \" + cfg.fmt)\n> +\n> + pix = rgb_to_pix(rgb)\n> +\n> + return pix\n> +\n> + def handle_request(self, stream, fb):\n> + ctx = self.ctx\n> +\n> + pix = self.buf_to_qpixmap(stream, fb)\n> + self.label.setPixmap(pix)\n> +\n> + self.frameLabel.setText(\"Queued: {}\\nDone: {}\\nFps: {:.2f}\"\n> + .format(ctx[\"reqs-queued\"], ctx[\"reqs-completed\"], ctx[\"fps\"]))\n> diff --git a/src/py/test/cam_qtgl.py b/src/py/test/cam_qtgl.py\n> new file mode 100644\n> index 00000000..01168979\n> --- /dev/null\n> +++ b/src/py/test/cam_qtgl.py\n> @@ -0,0 +1,385 @@\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +\n> +from PyQt5 import QtCore, QtWidgets\n> +from PyQt5.QtCore import Qt\n> +\n> +import numpy as np\n> +import sys\n> +import os\n> +os.environ[\"PYOPENGL_PLATFORM\"] = \"egl\"\n> +import math\n> +\n> +import OpenGL\n> +#OpenGL.FULL_LOGGING = True\n> +\n> +from OpenGL import GL as gl\n> +from OpenGL.EGL.KHR.image import *\n> +from OpenGL.EGL.EXT.image_dma_buf_import import *\n> +from OpenGL.EGL.VERSION.EGL_1_0 import *\n> +from OpenGL.EGL.VERSION.EGL_1_2 import *\n> +from OpenGL.EGL.VERSION.EGL_1_3 import *\n> +\n> +from OpenGL.GLES2.VERSION.GLES2_2_0 import *\n> +from OpenGL.GLES2.OES.EGL_image import *\n> +from OpenGL.GLES2.OES.EGL_image_external import *\n> +from OpenGL.GLES3.VERSION.GLES3_3_0 import *\n> +\n> +from OpenGL.GL import shaders\n> +\n> +from gl_helpers import *\n> +\n> +# libcamera format string -> DRM fourcc\n> +FMT_MAP = {\n> + \"RGB888\": \"RG24\",\n> + \"XRGB8888\": \"XR24\",\n> + \"ARGB8888\": \"AR24\",\n> + \"YUYV\": \"YUYV\",\n> +}\n> +\n> +class EglState:\n> + def __init__(self):\n> + self.create_display()\n> + self.choose_config()\n> + self.create_context()\n> + self.check_extensions()\n> +\n> + def create_display(self):\n> + xdpy = getEGLNativeDisplay()\n> + dpy = eglGetDisplay(xdpy)\n> + self.display = dpy\n> +\n> + def choose_config(self):\n> + dpy = self.display\n> +\n> + major, minor = EGLint(), EGLint()\n> +\n> + b = eglInitialize(dpy, major, minor)\n> + assert(b)\n> +\n> + print(\"EGL {} {}\".format(\n> + eglQueryString(dpy, EGL_VENDOR).decode(),\n> + eglQueryString(dpy, EGL_VERSION).decode()))\n> +\n> + check_egl_extensions(dpy, [\"EGL_EXT_image_dma_buf_import\"])\n> +\n> + b = eglBindAPI(EGL_OPENGL_ES_API)\n> + assert(b)\n> +\n> + def print_config(dpy, cfg):\n> +\n> + def _getconf(dpy, cfg, a):\n> + value = ctypes.c_long()\n> + eglGetConfigAttrib(dpy, cfg, a, value)\n> + return value.value\n> +\n> + getconf = lambda a: _getconf(dpy, cfg, a)\n> +\n> + print(\"EGL Config {}: color buf {}/{}/{}/{} = {}, depth {}, stencil {}, native visualid {}, native visualtype {}\".format(\n> + getconf(EGL_CONFIG_ID),\n> + getconf(EGL_ALPHA_SIZE),\n> + getconf(EGL_RED_SIZE),\n> + getconf(EGL_GREEN_SIZE),\n> + getconf(EGL_BLUE_SIZE),\n> + getconf(EGL_BUFFER_SIZE),\n> + getconf(EGL_DEPTH_SIZE),\n> + getconf(EGL_STENCIL_SIZE),\n> + getconf(EGL_NATIVE_VISUAL_ID),\n> + getconf(EGL_NATIVE_VISUAL_TYPE)))\n> +\n> + if False:\n> + num_configs = ctypes.c_long()\n> + eglGetConfigs(dpy, None, 0, num_configs)\n> + print(\"{} configs\".format(num_configs.value))\n> +\n> + configs = (EGLConfig * num_configs.value)()\n> + eglGetConfigs(dpy, configs, num_configs.value, num_configs)\n> + for config_id in configs:\n> + print_config(dpy, config_id)\n> +\n> +\n> + config_attribs = [\n> + EGL_SURFACE_TYPE, EGL_WINDOW_BIT,\n> + EGL_RED_SIZE, 8,\n> + EGL_GREEN_SIZE, 8,\n> + EGL_BLUE_SIZE, 8,\n> + EGL_ALPHA_SIZE, 0,\n> + EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT,\n> + EGL_NONE,\n> + ]\n> +\n> + n = EGLint()\n> + configs = (EGLConfig * 1)()\n> + b = eglChooseConfig(dpy, config_attribs, configs, 1, n)\n> + assert(b and n.value == 1)\n> + config = configs[0]\n> +\n> + print(\"Chosen Config:\")\n> + print_config(dpy, config)\n> +\n> + self.config = config\n> +\n> + def create_context(self):\n> + dpy = self.display\n> +\n> + context_attribs = [\n> + EGL_CONTEXT_CLIENT_VERSION, 2,\n> + EGL_NONE,\n> + ]\n> +\n> + context = eglCreateContext(dpy, self.config, EGL_NO_CONTEXT, context_attribs)\n> + assert(context)\n> +\n> + b = eglMakeCurrent(dpy, EGL_NO_SURFACE, EGL_NO_SURFACE, context)\n> + assert(b)\n> +\n> + self.context = context\n> +\n> + def check_extensions(self):\n> + check_gl_extensions([\"GL_OES_EGL_image\"])\n> +\n> + assert(eglCreateImageKHR)\n> + assert(eglDestroyImageKHR)\n> + assert(glEGLImageTargetTexture2DOES)\n> +\n> +\n> +class QtRenderer:\n> + def __init__(self, state):\n> + self.state = state\n> +\n> + def setup(self):\n> + self.app = QtWidgets.QApplication([])\n> +\n> + window = MainWindow(self.state)\n> + window.setAttribute(QtCore.Qt.WA_ShowWithoutActivating)\n> + window.show()\n> +\n> + self.window = window\n> +\n> + def run(self):\n> + camnotif = QtCore.QSocketNotifier(self.state[\"cm\"].efd, QtCore.QSocketNotifier.Read)\n> + camnotif.activated.connect(lambda x: self.readcam())\n> +\n> + keynotif = QtCore.QSocketNotifier(sys.stdin.fileno(), QtCore.QSocketNotifier.Read)\n> + keynotif.activated.connect(lambda x: self.readkey())\n> +\n> + print(\"Capturing...\")\n> +\n> + self.app.exec()\n> +\n> + print(\"Exiting...\")\n> +\n> + def readcam(self):\n> + running = self.state[\"event_handler\"](self.state)\n> +\n> + if not running:\n> + self.app.quit()\n> +\n> + def readkey(self):\n> + sys.stdin.readline()\n> + self.app.quit()\n> +\n> + def request_handler(self, ctx, req):\n> + self.window.handle_request(ctx, req)\n> +\n> + def cleanup(self):\n> + self.window.close()\n> +\n> +\n> +class MainWindow(QtWidgets.QWidget):\n> + def __init__(self, state):\n> + super().__init__()\n> +\n> + self.setAttribute(Qt.WA_PaintOnScreen)\n> + self.setAttribute(Qt.WA_NativeWindow)\n> +\n> + self.state = state\n> +\n> + self.textures = {}\n> + self.reqqueue = {}\n> + self.current = {}\n> +\n> + for ctx in self.state[\"contexts\"]:\n> +\n> + self.reqqueue[ctx[\"idx\"]] = []\n> + self.current[ctx[\"idx\"]] = []\n> +\n> + for stream in ctx[\"streams\"]:\n> + fmt = stream.configuration.fmt\n> + size = stream.configuration.size\n> +\n> + if not fmt in FMT_MAP:\n> + raise Exception(\"Unsupported pixel format: \" + str(fmt))\n> +\n> + self.textures[stream] = None\n> +\n> + num_tiles = len(self.textures)\n> + self.num_columns = math.ceil(math.sqrt(num_tiles))\n> + self.num_rows = math.ceil(num_tiles / self.num_columns)\n> +\n> + self.egl = EglState()\n> +\n> + self.surface = None\n> +\n> + def paintEngine(self):\n> + return None\n> +\n> + def create_surface(self):\n> + native_surface = c_void_p(self.winId().__int__())\n> + surface = eglCreateWindowSurface(self.egl.display, self.egl.config,\n> + native_surface, None)\n> +\n> + b = eglMakeCurrent(self.egl.display, self.surface, self.surface, self.egl.context)\n> + assert(b)\n> +\n> + self.surface = surface\n> +\n> + def init_gl(self):\n> + self.create_surface()\n> +\n> + vertShaderSrc = \"\"\"\n> + attribute vec2 aPosition;\n> + varying vec2 texcoord;\n> +\n> + void main()\n> + {\n> + gl_Position = vec4(aPosition * 2.0 - 1.0, 0.0, 1.0);\n> + texcoord.x = aPosition.x;\n> + texcoord.y = 1.0 - aPosition.y;\n> + }\n> + \"\"\"\n> + fragShaderSrc = \"\"\"\n> + #extension GL_OES_EGL_image_external : enable\n> + precision mediump float;\n> + varying vec2 texcoord;\n> + uniform samplerExternalOES texture;\n> +\n> + void main()\n> + {\n> + gl_FragColor = texture2D(texture, texcoord);\n> + }\n> + \"\"\"\n> +\n> + program = shaders.compileProgram(\n> + shaders.compileShader(vertShaderSrc, GL_VERTEX_SHADER),\n> + shaders.compileShader(fragShaderSrc, GL_FRAGMENT_SHADER)\n> + )\n> +\n> + glUseProgram(program)\n> +\n> + glClearColor(0.5, 0.8, 0.7, 1.0)\n> +\n> + vertPositions = [\n> + 0.0, 0.0,\n> + 1.0, 0.0,\n> + 1.0, 1.0,\n> + 0.0, 1.0\n> + ]\n> +\n> + inputAttrib = glGetAttribLocation(program, \"aPosition\")\n> + glVertexAttribPointer(inputAttrib, 2, GL_FLOAT, GL_FALSE, 0, vertPositions)\n> + glEnableVertexAttribArray(inputAttrib)\n> +\n> +\n> + def create_texture(self, stream, fb):\n> + cfg = stream.configuration\n> + fmt = cfg.fmt\n> + fmt = str_to_fourcc(FMT_MAP[fmt])\n> + w, h = cfg.size\n> +\n> + attribs = [\n> + EGL_WIDTH, w,\n> + EGL_HEIGHT, h,\n> + EGL_LINUX_DRM_FOURCC_EXT, fmt,\n> + EGL_DMA_BUF_PLANE0_FD_EXT, fb.fd(0),\n> + EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,\n> + EGL_DMA_BUF_PLANE0_PITCH_EXT, cfg.stride,\n> + EGL_NONE,\n> + ]\n> +\n> + image = eglCreateImageKHR(self.egl.display,\n> + EGL_NO_CONTEXT,\n> + EGL_LINUX_DMA_BUF_EXT,\n> + None,\n> + attribs)\n> + assert(image)\n> +\n> + textures = glGenTextures(1)\n> + glBindTexture(GL_TEXTURE_EXTERNAL_OES, textures)\n> + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR)\n> + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR)\n> + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)\n> + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)\n> + glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, image)\n> +\n> + return textures\n> +\n> + def resizeEvent(self, event):\n> + size = event.size()\n> +\n> + print(\"Resize\", size)\n> +\n> + super().resizeEvent(event)\n> +\n> + if self.surface == None:\n> + return\n> +\n> + glViewport(0, 0, size.width()//2, size.height())\n> +\n> + def paintEvent(self, event):\n> + if self.surface == None:\n> + self.init_gl()\n> +\n> + for ctx_idx, queue in self.reqqueue.items():\n> + if len(queue) == 0:\n> + continue\n> +\n> + ctx = next(ctx for ctx in self.state[\"contexts\"] if ctx[\"idx\"] == ctx_idx)\n> +\n> + if self.current[ctx_idx]:\n> + old = self.current[ctx_idx]\n> + self.current[ctx_idx] = None\n> + self.state[\"request_prcessed\"](ctx, old)\n> +\n> + next_req = queue.pop(0)\n> + self.current[ctx_idx] = next_req\n> +\n> + stream, fb = next(iter(next_req.buffers.items()))\n> +\n> + self.textures[stream] = self.create_texture(stream, fb)\n> +\n> + self.paint_gl()\n> +\n> + def paint_gl(self):\n> + b = eglMakeCurrent(self.egl.display, self.surface, self.surface, self.egl.context)\n> + assert(b)\n> +\n> + glClear(GL_COLOR_BUFFER_BIT)\n> +\n> + size = self.size()\n> +\n> + for idx,ctx in enumerate(self.state[\"contexts\"]):\n> + for stream in ctx[\"streams\"]:\n> + if self.textures[stream] == None:\n> + continue\n> +\n> + w = size.width() // self.num_columns\n> + h = size.height() // self.num_rows\n> +\n> + x = idx % self.num_columns\n> + y = idx // self.num_columns\n> +\n> + x *= w\n> + y *= h\n> +\n> + glViewport(x, y, w, h)\n> +\n> + glBindTexture(GL_TEXTURE_EXTERNAL_OES, self.textures[stream])\n> + glDrawArrays(GL_TRIANGLE_FAN, 0, 4)\n> +\n> + b = eglSwapBuffers(self.egl.display, self.surface)\n> + assert(b)\n> +\n> + def handle_request(self, ctx, req):\n> + self.reqqueue[ctx[\"idx\"]].append(req)\n> + self.update()\n> diff --git a/src/py/test/gl_helpers.py b/src/py/test/gl_helpers.py\n> new file mode 100644\n> index 00000000..a80b03b2\n> --- /dev/null\n> +++ b/src/py/test/gl_helpers.py\n> @@ -0,0 +1,67 @@\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2021, Tomi Valkeinen \n> +\n> +from OpenGL.EGL.VERSION.EGL_1_0 import EGLNativeDisplayType, eglGetProcAddress, eglQueryString, EGL_EXTENSIONS\n> +\n> +from OpenGL.raw.GLES2 import _types as _cs\n> +from OpenGL.GLES2.VERSION.GLES2_2_0 import *\n> +from OpenGL.GLES3.VERSION.GLES3_3_0 import *\n> +from OpenGL import GL as gl\n> +\n> +from ctypes import c_int, c_char_p, c_void_p, cdll, POINTER, util, \\\n> + pointer, CFUNCTYPE, c_bool\n> +\n> +def getEGLNativeDisplay():\n> + _x11lib = cdll.LoadLibrary(util.find_library(\"X11\"))\n> + XOpenDisplay = _x11lib.XOpenDisplay\n> + XOpenDisplay.argtypes = [c_char_p]\n> + XOpenDisplay.restype = POINTER(EGLNativeDisplayType)\n> +\n> + xdpy = XOpenDisplay(None)\n> +\n> +# Hack. PyOpenGL doesn't seem to manage to find glEGLImageTargetTexture2DOES.\n> +def getglEGLImageTargetTexture2DOES():\n> + funcptr = eglGetProcAddress(\"glEGLImageTargetTexture2DOES\")\n> + prototype = CFUNCTYPE(None,_cs.GLenum,_cs.GLeglImageOES)\n> + return prototype(funcptr)\n> +\n> +glEGLImageTargetTexture2DOES = getglEGLImageTargetTexture2DOES()\n> +\n> +\n> +def str_to_fourcc(str):\n> + assert(len(str) == 4)\n> + fourcc = 0\n> + for i,v in enumerate([ord(c) for c in str]):\n> + fourcc |= v << (i * 8)\n> + return fourcc\n> +\n> +def get_gl_extensions():\n> + n = GLint()\n> + glGetIntegerv(GL_NUM_EXTENSIONS, n)\n> + gl_extensions = []\n> + for i in range(n.value):\n> + gl_extensions.append(gl.glGetStringi(GL_EXTENSIONS, i).decode())\n> + return gl_extensions\n> +\n> +def check_gl_extensions(required_extensions):\n> + extensions = get_gl_extensions()\n> +\n> + if False:\n> + print(\"GL EXTENSIONS: \", \" \".join(extensions))\n> +\n> + for ext in required_extensions:\n> + if not ext in extensions:\n> + raise Exception(ext + \" missing\")\n> +\n> +def get_egl_extensions(egl_display):\n> + return eglQueryString(egl_display, EGL_EXTENSIONS).decode().split(\" \")\n> +\n> +def check_egl_extensions(egl_display, required_extensions):\n> + extensions = get_egl_extensions(egl_display)\n> +\n> + if False:\n> + print(\"EGL EXTENSIONS: \", \" \".join(extensions))\n> +\n> + for ext in required_extensions:\n> + if not ext in extensions:\n> + raise Exception(ext + \" missing\")\n> -- \n> 2.25.1\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9394DBF415\n\tfor ;\n\tThu, 9 Dec 2021 10:03:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A8B4A60882;\n\tThu, 9 Dec 2021 11:03:39 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3581160224\n\tfor ;\n\tThu, 9 Dec 2021 11:03:38 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AA435501;\n\tThu, 9 Dec 2021 11:03:37 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"rp3X+6H0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1639044217;\n\tbh=v1ur0C3y8TNXjhaBGL9U38La3/+Lid+vj4Il77UpMcU=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=rp3X+6H0BjNo4OHPIHh4T+6OnoaB3Z4KrY+PyNCaSQNGUjF1EQ3qEEYFNsVKehcqx\n\tTFUYHSjph1845ILjyzQWzl9K3HGpxzskqKiwbB2slNHOKacG04SeteIjwIldTXZ6SD\n\tDKZvT1N1wKnKeBhN6heqapV9HxRf+h9qMM6PtAtw=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>","References":"<20211209092906.37303-1-tomi.valkeinen@ideasonboard.com>\n\t<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>","From":"Kieran Bingham ","To":"Tomi Valkeinen ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Thu, 09 Dec 2021 10:03:35 +0000","Message-ID":"<163904421518.2066819.4960395426215054722@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":21696,"web_url":"https://patchwork.libcamera.org/comment/21696/","msgid":"<8eb9eef1-8845-4797-c2e2-181674139d3c@ideasonboard.com>","date":"2021-12-09T10:08:20","subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","submitter":{"id":109,"url":"https://patchwork.libcamera.org/api/people/109/","name":"Tomi Valkeinen","email":"tomi.valkeinen@ideasonboard.com"},"content":"On 09/12/2021 12:03, Kieran Bingham wrote:\n\n> I'd quite like to see something (anything?) that counts as a unittest\n> integrated into /test/py/ that somehow validates the APIs that are\n> created.\n> \n> Even if it's only basic for now to construct a camera, or something, so\n> that when built/enabled, the python code can be linked into the test\n> framework with 'ninja -C build test'.\n\nI've got this:\n\nhttps://github.com/tomba/libcamera/blob/py/src/py/test/unittests.py\n\nI probably could have added it in the series.\n\nHow do you test libcamera with unittests? With vimc?\n\n Tomi","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3544FBDB13\n\tfor ;\n\tThu, 9 Dec 2021 10:08:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DB47660822;\n\tThu, 9 Dec 2021 11:08:24 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 12F8760224\n\tfor ;\n\tThu, 9 Dec 2021 11:08:23 +0100 (CET)","from [192.168.1.111] (91-156-85-209.elisa-laajakaista.fi\n\t[91.156.85.209])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AA083501;\n\tThu, 9 Dec 2021 11:08:22 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Z7xyqXts\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1639044502;\n\tbh=k1ouojwuqPZiBQxVR87pzdL/G/bGwWpYzeLX117dhR8=;\n\th=Subject:To:References:From:Date:In-Reply-To:From;\n\tb=Z7xyqXtstnfTq1DykukSbrkYjAP0R64CLewIzdO4exsfbeFu4BDH1LrZnN/DRBevv\n\t/A7NFNJ/s1Ug/lU2XYQGILqSk/BlGUBYviXfR0vXtd8E1DHIunCLDOWAYJjtJoqx7S\n\tDiXShs5hRvAkEBfWlhjXoyuQrHfj+MAHcM2PX+v4=","To":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20211209092906.37303-1-tomi.valkeinen@ideasonboard.com>\n\t<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>\n\t<163904421518.2066819.4960395426215054722@Monstersaurus>","From":"Tomi Valkeinen ","Message-ID":"<8eb9eef1-8845-4797-c2e2-181674139d3c@ideasonboard.com>","Date":"Thu, 9 Dec 2021 12:08:20 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.14.0","MIME-Version":"1.0","In-Reply-To":"<163904421518.2066819.4960395426215054722@Monstersaurus>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":21698,"web_url":"https://patchwork.libcamera.org/comment/21698/","msgid":"<163904491584.2066819.4012988517911279617@Monstersaurus>","date":"2021-12-09T10:15:15","subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Tomi Valkeinen (2021-12-09 10:08:20)\n> On 09/12/2021 12:03, Kieran Bingham wrote:\n> \n> > I'd quite like to see something (anything?) that counts as a unittest\n> > integrated into /test/py/ that somehow validates the APIs that are\n> > created.\n> > \n> > Even if it's only basic for now to construct a camera, or something, so\n> > that when built/enabled, the python code can be linked into the test\n> > framework with 'ninja -C build test'.\n> \n> I've got this:\n> \n> https://github.com/tomba/libcamera/blob/py/src/py/test/unittests.py\n> \n> I probably could have added it in the series.\n> \n> How do you test libcamera with unittests? With vimc?\n\nPrecisely ;-)\n\nhttps://github.com/tomba/libcamera/blob/py/test/camera/capture.cpp#L28\n\nAnd it's perfectly valid to return that a test is Skipped if it can't\nfind a camera device (if it expected to need one).\n\nThe unit tests are defined as needing the following to run I believe:\n modprobe vimc\n modprobe vim2m\n\n> Tomi","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 876D3BF415\n\tfor ;\n\tThu, 9 Dec 2021 10:15:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 38E3560876;\n\tThu, 9 Dec 2021 11:15:20 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 873E360224\n\tfor ;\n\tThu, 9 Dec 2021 11:15:18 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 30BF2501;\n\tThu, 9 Dec 2021 11:15:18 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"D+heW/0F\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1639044918;\n\tbh=LdW5XSFdZ/EVhihLIT9ktJUrTI2b4g3X0Q2aCeMXVpA=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=D+heW/0F22rVxpkumexJhYPs4i2JWAC31b58eFjsgssr6ggjKJXBI3m/0van1Tzhf\n\ti8IXVbDathRjNuijNv3oxuzWWcepGCLwz0ANlQumqyXo+XbIDto8cAvDqqIH7Pal1L\n\tfejRefPvRTAl5S6u0ER2Kj3j0UE6JmqXjoWH++t4=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<8eb9eef1-8845-4797-c2e2-181674139d3c@ideasonboard.com>","References":"<20211209092906.37303-1-tomi.valkeinen@ideasonboard.com>\n\t<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>\n\t<163904421518.2066819.4960395426215054722@Monstersaurus>\n\t<8eb9eef1-8845-4797-c2e2-181674139d3c@ideasonboard.com>","From":"Kieran Bingham ","To":"Tomi Valkeinen ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Thu, 09 Dec 2021 10:15:15 +0000","Message-ID":"<163904491584.2066819.4012988517911279617@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":21720,"web_url":"https://patchwork.libcamera.org/comment/21720/","msgid":"","date":"2021-12-09T19:30:00","subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Thu, Dec 09, 2021 at 10:15:15AM +0000, Kieran Bingham wrote:\n> Quoting Tomi Valkeinen (2021-12-09 10:08:20)\n> > On 09/12/2021 12:03, Kieran Bingham wrote:\n> > \n> > > I'd quite like to see something (anything?) that counts as a unittest\n> > > integrated into /test/py/ that somehow validates the APIs that are\n> > > created.\n> > > \n> > > Even if it's only basic for now to construct a camera, or something, so\n> > > that when built/enabled, the python code can be linked into the test\n> > > framework with 'ninja -C build test'.\n> > \n> > I've got this:\n> > \n> > https://github.com/tomba/libcamera/blob/py/src/py/test/unittests.py\n> > \n> > I probably could have added it in the series.\n> > \n> > How do you test libcamera with unittests? With vimc?\n> \n> Precisely ;-)\n> \n> https://github.com/tomba/libcamera/blob/py/test/camera/capture.cpp#L28\n> \n> And it's perfectly valid to return that a test is Skipped if it can't\n> find a camera device (if it expected to need one).\n> \n> The unit tests are defined as needing the following to run I believe:\n> modprobe vimc\n> modprobe vim2m\n\nWe also depend on vivid.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6B6AEBF415\n\tfor ;\n\tThu, 9 Dec 2021 19:30:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 206F16086A;\n\tThu, 9 Dec 2021 20:30:31 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AB916607DE\n\tfor ;\n\tThu, 9 Dec 2021 20:30:29 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4638F501;\n\tThu, 9 Dec 2021 20:30:29 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"V0DAzWLA\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1639078229;\n\tbh=/7QMIDQqVkaNQ6E/S759K90IOhDWWhvNC+4MM05cP9I=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=V0DAzWLAFBc3mJ5ElOsn0bQSiWHIMdqzEeHz2nsSyGdIV3d3tY9s21OOh/LQyEW6s\n\tZKseDItDsqk+pV7DDLqBDvY4S9LFgenZJBziH6ArdXcvuFl7wgwyGKTZdiX/re0EsZ\n\tEoO3kZ0+RT9E5DvjwvE9Fp7ZXNNf52vnqtyyzvUU=","Date":"Thu, 9 Dec 2021 21:30:00 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"","References":"<20211209092906.37303-1-tomi.valkeinen@ideasonboard.com>\n\t<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>\n\t<163904421518.2066819.4960395426215054722@Monstersaurus>\n\t<8eb9eef1-8845-4797-c2e2-181674139d3c@ideasonboard.com>\n\t<163904491584.2066819.4012988517911279617@Monstersaurus>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<163904491584.2066819.4012988517911279617@Monstersaurus>","Subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":21721,"web_url":"https://patchwork.libcamera.org/comment/21721/","msgid":"","date":"2021-12-09T19:44:13","subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Thu, Dec 09, 2021 at 10:03:35AM +0000, Kieran Bingham wrote:\n> Quoting Tomi Valkeinen (2021-12-09 09:29:06)\n> > Add cam.py, which mimics the 'cam' tool. Four rendering backends are\n> > added:\n> > \n> > * null - Do nothing\n> > * kms - Use KMS with dmabufs\n> > * qt - SW render on a Qt window\n> > * qtgl - OpenGL render on a Qt window\n> > \n> > All the renderers handle only a few pixel formats, and especially the GL\n> > renderer is just a prototype.\n> > \n> > Signed-off-by: Tomi Valkeinen \n> > ---\n> > src/py/test/cam.py | 464 ++++++++++++++++++++++++++++++++++++++\n> > src/py/test/cam_kms.py | 185 +++++++++++++++\n> > src/py/test/cam_null.py | 46 ++++\n> > src/py/test/cam_qt.py | 355 +++++++++++++++++++++++++++++\n> > src/py/test/cam_qtgl.py | 385 +++++++++++++++++++++++++++++++\n> > src/py/test/gl_helpers.py | 67 ++++++\n> \n> While these are 'test' apps, they're not quite 'tests'.\n> \n> I think these should live under src/py/pycam? Or some such name?\n\nLooks good to me.\n\n> Essentially they will likely form the basis of 'example python usage'...\n> \n> I'd quite like to see something (anything?) that counts as a unittest\n> integrated into /test/py/ that somehow validates the APIs that are\n> created.\n\nDitto.\n\n> Even if it's only basic for now to construct a camera, or something, so\n> that when built/enabled, the python code can be linked into the test\n> framework with 'ninja -C build test'.\n> \n> I wonder if a top-level TODO file under src/py/TODO and/or\n> src/py/{test/pycam}/TODO will help clearly mark things that are known\n> not to be implemented ?\n> \n> \n> Beyond that, I think this series would benefit from early integration so\n> it can be more thoroughly used and devloped rather than trying to\n> 'perfect' it out in isolation.\n\nI agree here too. There's a set of small comments that should be\naddressed in a v4 for the parts that touch the libcamera C++ API, the\nbuild system or .gitignore, for the Python side we can then work on top\nof this series after merging it.\n\n> > 6 files changed, 1502 insertions(+)\n> > create mode 100755 src/py/test/cam.py\n> > create mode 100644 src/py/test/cam_kms.py\n> > create mode 100644 src/py/test/cam_null.py\n> > create mode 100644 src/py/test/cam_qt.py\n> > create mode 100644 src/py/test/cam_qtgl.py\n> > create mode 100644 src/py/test/gl_helpers.py\n> > \n> > diff --git a/src/py/test/cam.py b/src/py/test/cam.py\n> > new file mode 100755\n> > index 00000000..48df01cf\n> > --- /dev/null\n> > +++ b/src/py/test/cam.py\n> > @@ -0,0 +1,464 @@\n> > +#!/usr/bin/python3\n\nIsn't\n\n#!/usr/bin/env python3\n\npreferred, to support custom Python installation ?\n\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +\n> > +import pycamera as pycam\n> > +import time\n> > +import binascii\n> > +import argparse\n> > +import selectors\n> > +import os\n> > +import sys\n\nI think Python too knows about alphabetical ordering ;-)\n\n> > +\n> > +class CustomCameraAction(argparse.Action):\n> > + def __call__(self, parser, namespace, values, option_string=None):\n> > + print(self.dest, values)\n> > +\n> > + if not \"camera\" in namespace or namespace.camera == None:\n> > + setattr(namespace, \"camera\", [])\n> > +\n> > + previous = namespace.camera\n> > + previous.append((self.dest, values))\n> > + setattr(namespace, \"camera\", previous)\n> > +\n> > +class CustomAction(argparse.Action):\n> > + def __init__(self, option_strings, dest, **kwargs):\n> > + super().__init__(option_strings, dest, default={}, **kwargs)\n> > +\n> > + def __call__(self, parser, namespace, values, option_string=None):\n> > + if len(namespace.camera) == 0:\n> > + print(f\"Option {option_string} requires a --camera context\")\n> > + sys.exit(-1)\n> > +\n> > + if self.type == bool:\n> > + values = True\n> > +\n> > + current = namespace.camera[-1]\n> > +\n> > + data = getattr(namespace, self.dest)\n> > +\n> > + if self.nargs == \"+\":\n> > + if not current in data:\n> > + data[current] = []\n> > +\n> > + data[current] += values\n> > + else:\n> > + data[current] = values\n> > +\n> > +\n> > +\n> > +def do_cmd_list(cm):\n> > + print(\"Available cameras:\")\n> > +\n> > + for idx,c in enumerate(cm.cameras):\n> > + print(f\"{idx + 1}: {c.id}\")\n> > +\n\nIt looks like lots of the functions below that take a context argument\ncould be moved to a context class.\n\n> > +def do_cmd_list_props(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + print(\"Properties for\", ctx[\"id\"])\n> > +\n> > + for name, prop in camera.properties.items():\n> > + print(\"\\t{}: {}\".format(name, prop))\n> > +\n> > +def do_cmd_list_controls(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + print(\"Controls for\", ctx[\"id\"])\n> > +\n> > + for name, prop in camera.controls.items():\n> > + print(\"\\t{}: {}\".format(name, prop))\n> > +\n> > +def do_cmd_info(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + print(\"Stream info for\", ctx[\"id\"])\n> > +\n> > + roles = [pycam.StreamRole.Viewfinder]\n> > +\n> > + camconfig = camera.generateConfiguration(roles)\n> > + if camconfig == None:\n> > + raise Exception(\"Generating config failed\")\n> > +\n> > + for i, stream_config in enumerate(camconfig):\n> > + print(\"\\t{}: {}\".format(i, stream_config.toString()))\n> > +\n> > + formats = stream_config.formats\n> > + for fmt in formats.pixelFormats:\n> > + print(\"\\t * Pixelformat:\", fmt, formats.range(fmt))\n> > +\n> > + for size in formats.sizes(fmt):\n> > + print(\"\\t -\", size)\n> > +\n> > +def acquire(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + camera.acquire()\n> > +\n> > +def release(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + camera.release()\n> > +\n> > +def parse_streams(ctx):\n> > + streams = []\n> > +\n> > + for stream_desc in ctx[\"opt-stream\"]:\n> > + stream_opts = {\"role\": pycam.StreamRole.Viewfinder}\n> > +\n> > + for stream_opt in stream_desc.split(\",\"):\n> > + if stream_opt == 0:\n> > + continue\n> > +\n> > + arr = stream_opt.split(\"=\")\n> > + if len(arr) != 2:\n> > + print(\"Bad stream option\", stream_opt)\n> > + sys.exit(-1)\n> > +\n> > + key = arr[0]\n> > + value = arr[1]\n> > +\n> > + if key in [\"width\", \"height\"]:\n> > + value = int(value)\n> > + elif key == \"role\":\n> > + rolemap = {\n> > + \"still\": pycam.StreamRole.StillCapture,\n> > + \"raw\": pycam.StreamRole.Raw,\n> > + \"video\": pycam.StreamRole.VideoRecording,\n> > + \"viewfinder\": pycam.StreamRole.Viewfinder,\n> > + }\n> > +\n> > + role = rolemap.get(value.lower(), None)\n> > +\n> > + if role == None:\n> > + print(\"Bad stream role\", value)\n> > + sys.exit(-1)\n> > +\n> > + value = role\n> > + elif key == \"pixelformat\":\n> > + pass\n> > + else:\n> > + print(\"Bad stream option key\", key)\n> > + sys.exit(-1)\n> > +\n> > + stream_opts[key] = value\n> > +\n> > + streams.append(stream_opts)\n> > +\n> > + return streams\n> > +\n> > +def configure(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + streams = parse_streams(ctx)\n> > +\n> > + roles = [opts[\"role\"] for opts in streams]\n> > +\n> > + camconfig = camera.generateConfiguration(roles)\n> > + if camconfig == None:\n> > + raise Exception(\"Generating config failed\")\n> > +\n> > + for idx,stream_opts in enumerate(streams):\n> > + stream_config = camconfig.at(idx)\n> > +\n> > + if \"width\" in stream_opts and \"height\" in stream_opts:\n> > + stream_config.size = (stream_opts[\"width\"], stream_opts[\"height\"])\n> > +\n> > + if \"pixelformat\" in stream_opts:\n> > + stream_config.fmt = stream_opts[\"pixelformat\"]\n> > +\n> > + stat = camconfig.validate()\n> > +\n> > + if stat == pycam.ConfigurationStatus.Invalid:\n> > + print(\"Camera configuration invalid\")\n> > + exit(-1)\n> > + elif stat == pycam.ConfigurationStatus.Adjusted:\n> > + if ctx[\"opt-strict-formats\"]:\n> > + print(\"Adjusting camera configuration disallowed by --strict-formats argument\")\n> > + exit(-1)\n> > +\n> > + print(\"Camera configuration adjusted\")\n> > +\n> > + r = camera.configure(camconfig);\n> > + if r != 0:\n> > + raise Exception(\"Configure failed\")\n> > +\n> > + ctx[\"stream-names\"] = {}\n> > + ctx[\"streams\"] = []\n> > +\n> > + for idx, stream_config in enumerate(camconfig):\n> > + stream = stream_config.stream\n> > + ctx[\"streams\"].append(stream)\n> > + ctx[\"stream-names\"][stream] = \"stream\" + str(idx)\n> > + print(\"{}-{}: stream config {}\".format(ctx[\"id\"], ctx[\"stream-names\"][stream], stream.configuration.toString()))\n> > +\n> > +def alloc_buffers(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + allocator = pycam.FrameBufferAllocator(camera);\n> > +\n> > + for idx, stream in enumerate(ctx[\"streams\"]):\n> > + ret = allocator.allocate(stream)\n> > + if ret < 0:\n> > + print(\"Can't allocate buffers\")\n> > + exit(-1)\n> > +\n> > + allocated = len(allocator.buffers(stream))\n> > +\n> > + print(\"{}-{}: Allocated {} buffers\".format(ctx[\"id\"], ctx[\"stream-names\"][stream], allocated))\n> > +\n> > + ctx[\"allocator\"] = allocator\n> > +\n> > +def create_requests(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + ctx[\"requests\"] = []\n> > +\n> > + # Identify the stream with the least number of buffers\n> > + num_bufs = min([len(ctx[\"allocator\"].buffers(stream)) for stream in ctx[\"streams\"]])\n> > +\n> > + requests = []\n> > +\n> > + for buf_num in range(num_bufs):\n> > + request = camera.createRequest()\n> > +\n> > + if request == None:\n> > + print(\"Can't create request\")\n> > + exit(-1)\n> > +\n> > + for stream in ctx[\"streams\"]:\n> > + buffers = ctx[\"allocator\"].buffers(stream)\n> > + buffer = buffers[buf_num]\n> > +\n> > + ret = request.addBuffer(stream, buffer)\n> > + if ret < 0:\n> > + print(\"Can't set buffer for request\")\n> > + exit(-1)\n> > +\n> > + requests.append(request)\n> > +\n> > + ctx[\"requests\"] = requests\n> > +\n> > +def start(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + camera.start()\n> > +\n> > +def stop(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + camera.stop()\n> > +\n> > +def queue_requests(ctx):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + for request in ctx[\"requests\"]:\n> > + camera.queueRequest(request)\n> > + ctx[\"reqs-queued\"] += 1\n> > +\n> > + del ctx[\"requests\"]\n> > +\n> > +def capture_init(contexts):\n> > + for ctx in contexts:\n> > + acquire(ctx)\n> > +\n> > + for ctx in contexts:\n> > + configure(ctx)\n> > +\n> > + for ctx in contexts:\n> > + alloc_buffers(ctx)\n> > +\n> > + for ctx in contexts:\n> > + create_requests(ctx)\n> > +\n> > +def capture_start(contexts):\n> > + for ctx in contexts:\n> > + start(ctx)\n> > +\n> > + for ctx in contexts:\n> > + queue_requests(ctx)\n> > +\n> > +# Called from renderer when there is a libcamera event\n> > +def event_handler(state):\n> > + cm = state[\"cm\"]\n> > + contexts = state[\"contexts\"]\n> > +\n> > + data = os.read(cm.efd, 8)\n> > +\n> > + reqs = cm.getReadyRequests()\n> > +\n> > + for req in reqs:\n> > + ctx = next(ctx for ctx in contexts if ctx[\"camera\"] == req.camera)\n> > + request_handler(state, ctx, req)\n> > +\n> > + running = any(ctx[\"reqs-completed\"] < ctx[\"opt-capture\"] for ctx in contexts)\n> > + return running\n> > +\n> > +def request_handler(state, ctx, req):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + if req.status != pycam.RequestStatus.Complete:\n> > + raise Exception(\"{}: Request failed: {}\".format(ctx[\"id\"], req.status))\n> > +\n> > + buffers = req.buffers\n> > +\n> > + # Compute the frame rate. The timestamp is arbitrarily retrieved from\n> > + # the first buffer, as all buffers should have matching timestamps.\n> > + ts = buffers[next(iter(buffers))].metadata.timestamp\n> > + last = ctx.get(\"last\", 0)\n> > + fps = 1000000000.0 / (ts - last) if (last != 0 and (ts - last) != 0) else 0\n> > + ctx[\"last\"] = ts\n> > + ctx[\"fps\"] = fps\n> > +\n> > + for stream, fb in buffers.items():\n> > + stream_name = ctx[\"stream-names\"][stream]\n> > +\n> > + crcs = []\n> > + if ctx[\"opt-crc\"]:\n> > + with fb.mmap(0) as b:\n> > + crc = binascii.crc32(b)\n> > + crcs.append(crc)\n> > +\n> > + meta = fb.metadata\n> > +\n> > + print(\"{:.6f} ({:.2f} fps) {}-{}: seq {}, bytes {}, CRCs {}\"\n> > + .format(ts / 1000000000, fps,\n> > + ctx[\"id\"], stream_name,\n> > + meta.sequence, meta.bytesused,\n> > + crcs))\n> > +\n> > + if ctx[\"opt-metadata\"]:\n> > + reqmeta = req.metadata\n> > + for ctrl, val in reqmeta.items():\n> > + print(f\"\\t{ctrl} = {val}\")\n> > +\n> > + if ctx[\"opt-save-frames\"]:\n> > + with fb.mmap(0) as b:\n> > + filename = \"frame-{}-{}-{}.data\".format(ctx[\"id\"], stream_name, ctx[\"reqs-completed\"])\n> > + with open(filename, \"wb\") as f:\n> > + f.write(b)\n> > +\n> > + state[\"renderer\"].request_handler(ctx, req);\n> > +\n> > + ctx[\"reqs-completed\"] += 1\n> > +\n> > +# Called from renderer when it has finished with a request\n> > +def request_prcessed(ctx, req):\n> > + camera = ctx[\"camera\"]\n> > +\n> > + if ctx[\"reqs-queued\"] < ctx[\"opt-capture\"]:\n> > + req.reuse()\n> > + camera.queueRequest(req)\n> > + ctx[\"reqs-queued\"] += 1\n> > +\n> > +def capture_deinit(contexts):\n> > + for ctx in contexts:\n> > + stop(ctx)\n> > +\n> > + for ctx in contexts:\n> > + release(ctx)\n> > +\n> > +def do_cmd_capture(state):\n> > + capture_init(state[\"contexts\"])\n> > +\n> > + renderer = state[\"renderer\"]\n> > +\n> > + renderer.setup()\n> > +\n> > + capture_start(state[\"contexts\"])\n> > +\n> > + renderer.run()\n> > +\n> > + capture_deinit(state[\"contexts\"])\n> > +\n> > +def main():\n> > + parser = argparse.ArgumentParser()\n> > + # global options\n> > + parser.add_argument(\"-l\", \"--list\", action=\"store_true\", help=\"List all cameras\")\n> > + parser.add_argument(\"-c\", \"--camera\", type=int, action=\"extend\", nargs=1, default=[], help=\"Specify which camera to operate on, by index\")\n> > + parser.add_argument(\"-p\", \"--list-properties\", action=\"store_true\", help=\"List cameras properties\")\n> > + parser.add_argument(\"--list-controls\", action=\"store_true\", help=\"List cameras controls\")\n> > + parser.add_argument(\"-I\", \"--info\", action=\"store_true\", help=\"Display information about stream(s)\")\n> > + parser.add_argument(\"-R\", \"--renderer\", default=\"null\", help=\"Renderer (null, kms, qt, qtgl)\")\n> > +\n> > + # per camera options\n> > + parser.add_argument(\"-C\", \"--capture\", nargs=\"?\", type=int, const=1000000, action=CustomAction, help=\"Capture until interrupted by user or until CAPTURE frames captured\")\n> > + parser.add_argument(\"--crc\", nargs=0, type=bool, action=CustomAction, help=\"Print CRC32 for captured frames\")\n> > + parser.add_argument(\"--save-frames\", nargs=0, type=bool, action=CustomAction, help=\"Save captured frames to files\")\n> > + parser.add_argument(\"--metadata\", nargs=0, type=bool, action=CustomAction, help=\"Print the metadata for completed requests\")\n> > + parser.add_argument(\"--strict-formats\", type=bool, nargs=0, action=CustomAction, help=\"Do not allow requested stream format(s) to be adjusted\")\n> > + parser.add_argument(\"-s\", \"--stream\", nargs=\"+\", action=CustomAction)\n> > + args = parser.parse_args()\n> > +\n> > + cm = pycam.CameraManager.singleton()\n> > +\n> > + if args.list:\n> > + do_cmd_list(cm)\n> > +\n> > + contexts = []\n> > +\n> > + for cam_idx in args.camera:\n> > + camera = next((c for i,c in enumerate(cm.cameras) if i + 1 == cam_idx), None)\n> > +\n> > + if camera == None:\n> > + print(\"Unable to find camera\", cam_idx)\n> > + return -1\n> > +\n> > + contexts.append({\n> > + \"camera\": camera,\n> > + \"idx\": cam_idx,\n> > + \"id\": \"cam\" + str(cam_idx),\n> > + \"reqs-queued\": 0,\n> > + \"reqs-completed\": 0,\n> > + \"opt-capture\": args.capture.get(cam_idx, False),\n> > + \"opt-crc\": args.crc.get(cam_idx, False),\n> > + \"opt-save-frames\": args.save_frames.get(cam_idx, False),\n> > + \"opt-metadata\": args.metadata.get(cam_idx, False),\n> > + \"opt-strict-formats\": args.strict_formats.get(cam_idx, False),\n> > + \"opt-stream\": args.stream.get(cam_idx, [\"role=viewfinder\"]),\n> > + })\n\nYes, this definitely looks like a candidate for a class :-)\n\n> > +\n> > + for ctx in contexts:\n> > + print(\"Using camera {} as {}\".format(ctx[\"camera\"].id, ctx[\"id\"]))\n> > +\n> > + for ctx in contexts:\n> > + if args.list_properties:\n> > + do_cmd_list_props(ctx)\n> > + if args.list_controls:\n> > + do_cmd_list_controls(ctx)\n> > + if args.info:\n> > + do_cmd_info(ctx)\n> > +\n> > + if args.capture:\n> > +\n> > + state = {\n> > + \"cm\": cm,\n> > + \"contexts\": contexts,\n> > + \"event_handler\": event_handler,\n> > + \"request_prcessed\": request_prcessed,\n> > + }\n> > +\n> > + if args.renderer == \"null\":\n> > + import cam_null\n> > + renderer = cam_null.NullRenderer(state)\n> > + elif args.renderer == \"kms\":\n> > + import cam_kms\n> > + renderer = cam_kms.KMSRenderer(state)\n> > + elif args.renderer == \"qt\":\n> > + import cam_qt\n> > + renderer = cam_qt.QtRenderer(state)\n\nInteresting, I would probably instinctively split this application in a\ncommand line version and a Qt version, but I suppose this makes sense\ntoo, as Python allows runtime linking.\n\n> > + elif args.renderer == \"qtgl\":\n> > + import cam_qtgl\n> > + renderer = cam_qtgl.QtRenderer(state)\n> > + else:\n> > + print(\"Bad renderer\", args.renderer)\n> > + return -1\n> > +\n> > + state[\"renderer\"] = renderer\n> > +\n> > + do_cmd_capture(state)\n> > +\n> > + return 0\n> > +\n> > +if __name__ == \"__main__\":\n> > + sys.exit(main())\n\nAn equivalent of simple-cam in Python would be useful too, cam.py is\nfairly large for a tutorial. This can be done later (even though I would\nhave imagined it would be the first application to be developed, as it\nwould be simpler).\n\n> > diff --git a/src/py/test/cam_kms.py b/src/py/test/cam_kms.py\n> > new file mode 100644\n> > index 00000000..fb0e6375\n> > --- /dev/null\n> > +++ b/src/py/test/cam_kms.py\n> > @@ -0,0 +1,185 @@\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +\n> > +import sys\n> > +import selectors\n> > +import pykms\n> > +\n> > +FMT_MAP = {\n> > + \"RGB888\": pykms.PixelFormat.RGB888,\n> > + \"YUYV\": pykms.PixelFormat.YUYV,\n> > + \"ARGB8888\": pykms.PixelFormat.ARGB8888,\n> > + \"XRGB8888\": pykms.PixelFormat.XRGB8888,\n> > +}\n> > +\n> > +class KMSRenderer:\n> > + def __init__(self, state):\n> > + self.state = state\n> > +\n> > + self.cm = state[\"cm\"]\n> > + self.contexts = state[\"contexts\"]\n> > + self.running = False\n> > +\n> > + card = pykms.Card()\n> > +\n> > + res = pykms.ResourceManager(card)\n> > + conn = res.reserve_connector()\n> > + crtc = res.reserve_crtc(conn)\n> > + mode = conn.get_default_mode()\n> > + modeb = mode.to_blob(card)\n> > +\n> > + req = pykms.AtomicReq(card)\n> > + req.add_connector(conn, crtc)\n> > + req.add_crtc(crtc, modeb)\n> > + r = req.commit_sync(allow_modeset = True)\n> > + assert(r == 0)\n> > +\n> > + self.card = card\n> > + self.resman = res\n> > + self.crtc = crtc\n> > + self.mode = mode\n> > +\n> > + self.bufqueue = []\n> > + self.current = None\n> > + self.next = None\n> > + self.cam_2_drm = {}\n> > +\n> > + # KMS\n> > +\n> > + def close(self):\n> > + req = pykms.AtomicReq(self.card)\n> > + for s in self.streams:\n> > + req.add_plane(s[\"plane\"], None, None, dst=(0, 0, 0, 0))\n> > + req.commit()\n> > +\n> > + def add_plane(self, req, stream, fb):\n> > + s = next(s for s in self.streams if s[\"stream\"] == stream)\n> > + idx = s[\"idx\"]\n> > + plane = s[\"plane\"]\n> > +\n> > + if idx % 2 == 0:\n> > + x = 0\n> > + else:\n> > + x = self.mode.hdisplay - fb.width\n> > +\n> > + if idx // 2 == 0:\n> > + y = 0\n> > + else:\n> > + y = self.mode.vdisplay - fb.height\n> > +\n> > + req.add_plane(plane, fb, self.crtc, dst=(x, y, fb.width, fb.height))\n> > +\n> > + def apply_request(self, drmreq):\n> > +\n> > + buffers = drmreq[\"camreq\"].buffers\n> > +\n> > + for stream, fb in buffers.items():\n> > + drmfb = self.cam_2_drm.get(fb, None)\n> > +\n> > + req = pykms.AtomicReq(self.card)\n> > + self.add_plane(req, stream, drmfb)\n> > + req.commit()\n> > +\n> > + def handle_page_flip(self, frame, time):\n> > + old = self.current\n> > + self.current = self.next\n> > +\n> > + if len(self.bufqueue) > 0:\n> > + self.next = self.bufqueue.pop(0)\n> > + else:\n> > + self.next = None\n> > +\n> > + if self.next:\n> > + drmreq = self.next\n> > +\n> > + self.apply_request(drmreq)\n> > +\n> > + if old:\n> > + req = old[\"camreq\"]\n> > + ctx = old[\"camctx\"]\n> > + self.state[\"request_prcessed\"](ctx, req)\n> > +\n> > + def queue(self, drmreq):\n> > + if not self.next:\n> > + self.next = drmreq\n> > + self.apply_request(drmreq)\n> > + else:\n> > + self.bufqueue.append(drmreq)\n> > +\n> > + # libcamera\n> > +\n> > + def setup(self):\n> > + self.streams = []\n> > +\n> > + idx = 0\n> > + for ctx in self.contexts:\n> > + camera = ctx[\"camera\"]\n> > +\n> > + for stream in ctx[\"streams\"]:\n> > +\n> > + cfg = stream.configuration\n> > + fmt = cfg.fmt\n> > + fmt = FMT_MAP[fmt]\n> > +\n> > + plane = self.resman.reserve_generic_plane(self.crtc, fmt)\n> > + assert(plane != None)\n> > +\n> > + self.streams.append({\n> > + \"idx\": idx,\n> > + \"stream\": stream,\n> > + \"plane\": plane,\n> > + \"fmt\": fmt,\n> > + \"size\": cfg.size,\n> > + })\n> > +\n> > + for fb in ctx[\"allocator\"].buffers(stream):\n> > + w, h = cfg.size\n> > + stride = cfg.stride\n> > + fd = fb.fd(0)\n> > + drmfb = pykms.DmabufFramebuffer(self.card, w, h, fmt,\n> > + [fd], [stride], [0])\n> > + self.cam_2_drm[fb] = drmfb\n> > +\n> > + idx += 1\n> > +\n> > +\n> > + def readdrm(self, fileobj):\n> > + for ev in self.card.read_events():\n> > + if ev.type == pykms.DrmEventType.FLIP_COMPLETE:\n> > + self.handle_page_flip(ev.seq, ev.time)\n> > +\n> > + def readcam(self, fd):\n> > + self.running = self.state[\"event_handler\"](self.state)\n> > +\n> > + def readkey(self, fileobj):\n> > + sys.stdin.readline()\n> > + self.running = False\n> > +\n> > + def run(self):\n> > + print(\"Capturing...\")\n> > +\n> > + self.running = True\n> > +\n> > + sel = selectors.DefaultSelector()\n> > + sel.register(self.card.fd, selectors.EVENT_READ, self.readdrm)\n> > + sel.register(self.cm.efd, selectors.EVENT_READ, self.readcam)\n> > + sel.register(sys.stdin, selectors.EVENT_READ, self.readkey)\n> > +\n> > + print(\"Press enter to exit\")\n> > +\n> > + while self.running:\n> > + events = sel.select()\n> > + for key, mask in events:\n> > + callback = key.data\n> > + callback(key.fileobj)\n> > +\n> > + print(\"Exiting...\")\n> > +\n> > + def request_handler(self, ctx, req):\n> > +\n> > + drmreq = {\n> > + \"camctx\": ctx,\n> > + \"camreq\": req,\n> > + }\n> > +\n> > + self.queue(drmreq)\n> > diff --git a/src/py/test/cam_null.py b/src/py/test/cam_null.py\n> > new file mode 100644\n> > index 00000000..3935f5b6\n> > --- /dev/null\n> > +++ b/src/py/test/cam_null.py\n> > @@ -0,0 +1,46 @@\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +\n> > +import sys\n> > +import selectors\n> > +\n> > +class NullRenderer:\n> > + def __init__(self, state):\n> > + self.state = state\n> > +\n> > + self.cm = state[\"cm\"]\n> > + self.contexts = state[\"contexts\"]\n> > +\n> > + self.running = False\n> > +\n> > + def setup(self):\n> > + pass\n> > +\n> > + def run(self):\n> > + print(\"Capturing...\")\n> > +\n> > + self.running = True\n> > +\n> > + sel = selectors.DefaultSelector()\n> > + sel.register(self.cm.efd, selectors.EVENT_READ, self.readcam)\n> > + sel.register(sys.stdin, selectors.EVENT_READ, self.readkey)\n> > +\n> > + print(\"Press enter to exit\")\n> > +\n> > + while self.running:\n> > + events = sel.select()\n> > + for key, mask in events:\n> > + callback = key.data\n> > + callback(key.fileobj)\n> > +\n> > + print(\"Exiting...\")\n> > +\n> > + def readcam(self, fd):\n> > + self.running = self.state[\"event_handler\"](self.state)\n> > +\n> > + def readkey(self, fileobj):\n> > + sys.stdin.readline()\n> > + self.running = False\n> > +\n> > + def request_handler(self, ctx, req):\n> > + self.state[\"request_prcessed\"](ctx, req)\n> > diff --git a/src/py/test/cam_qt.py b/src/py/test/cam_qt.py\n> > new file mode 100644\n> > index 00000000..3ff12df6\n> > --- /dev/null\n> > +++ b/src/py/test/cam_qt.py\n> > @@ -0,0 +1,355 @@\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +#\n> > +# Debayering code from PiCamera documentation\n> > +\n> > +from PyQt5 import QtCore, QtGui, QtWidgets\n> > +from io import BytesIO\n> > +from PIL import Image\n> > +from PIL.ImageQt import ImageQt\n> > +import numpy as np\n> > +from numpy.lib.stride_tricks import as_strided\n> > +import sys\n> > +\n> > +def rgb_to_pix(rgb):\n> > + img = Image.frombuffer(\"RGB\", (rgb.shape[1], rgb.shape[0]), rgb)\n> > + qim = ImageQt(img).copy()\n> > + pix = QtGui.QPixmap.fromImage(qim)\n> > + return pix\n> > +\n> > +\n> > +def separate_components(data, r0, g0, g1, b0):\n> > + # Now to split the data up into its red, green, and blue components. The\n> > + # Bayer pattern of the OV5647 sensor is BGGR. In other words the first\n> > + # row contains alternating green/blue elements, the second row contains\n> > + # alternating red/green elements, and so on as illustrated below:\n> > + #\n> > + # GBGBGBGBGBGBGB\n> > + # RGRGRGRGRGRGRG\n> > + # GBGBGBGBGBGBGB\n> > + # RGRGRGRGRGRGRG\n> > + #\n> > + # Please note that if you use vflip or hflip to change the orientation\n> > + # of the capture, you must flip the Bayer pattern accordingly\n> > +\n> > + rgb = np.zeros(data.shape + (3,), dtype=data.dtype)\n> > + rgb[r0[1]::2, r0[0]::2, 0] = data[r0[1]::2, r0[0]::2] # Red\n> > + rgb[g0[1]::2, g0[0]::2, 1] = data[g0[1]::2, g0[0]::2] # Green\n> > + rgb[g1[1]::2, g1[0]::2, 1] = data[g1[1]::2, g1[0]::2] # Green\n> > + rgb[b0[1]::2, b0[0]::2, 2] = data[b0[1]::2, b0[0]::2] # Blue\n> > +\n> > + return rgb\n> > +\n> > +def demosaic(rgb, r0, g0, g1, b0):\n\nI'd split the image processing out of cam_qt.py.\n\n> > + # At this point we now have the raw Bayer data with the correct values\n> > + # and colors but the data still requires de-mosaicing and\n> > + # post-processing. If you wish to do this yourself, end the script here!\n> > + #\n> > + # Below we present a fairly naive de-mosaic method that simply\n> > + # calculates the weighted average of a pixel based on the pixels\n> > + # surrounding it. The weighting is provided b0[1] a b0[1]te representation of\n> > + # the Bayer filter which we construct first:\n> > +\n> > + bayer = np.zeros(rgb.shape, dtype=np.uint8)\n> > + bayer[r0[1]::2, r0[0]::2, 0] = 1 # Red\n> > + bayer[g0[1]::2, g0[0]::2, 1] = 1 # Green\n> > + bayer[g1[1]::2, g1[0]::2, 1] = 1 # Green\n> > + bayer[b0[1]::2, b0[0]::2, 2] = 1 # Blue\n> > +\n> > + # Allocate an array to hold our output with the same shape as the input\n> > + # data. After this we define the size of window that will be used to\n> > + # calculate each weighted average (3x3). Then we pad out the rgb and\n> > + # bayer arrays, adding blank pixels at their edges to compensate for the\n> > + # size of the window when calculating averages for edge pixels.\n> > +\n> > + output = np.empty(rgb.shape, dtype=rgb.dtype)\n> > + window = (3, 3)\n> > + borders = (window[0] - 1, window[1] - 1)\n> > + border = (borders[0] // 2, borders[1] // 2)\n> > +\n> > + #rgb_pad = np.zeros((\n> > + # rgb.shape[0] + borders[0],\n> > + # rgb.shape[1] + borders[1],\n> > + # rgb.shape[2]), dtype=rgb.dtype)\n> > + #rgb_pad[\n> > + # border[0]:rgb_pad.shape[0] - border[0],\n> > + # border[1]:rgb_pad.shape[1] - border[1],\n> > + # :] = rgb\n> > + #rgb = rgb_pad\n> > + #\n> > + #bayer_pad = np.zeros((\n> > + # bayer.shape[0] + borders[0],\n> > + # bayer.shape[1] + borders[1],\n> > + # bayer.shape[2]), dtype=bayer.dtype)\n> > + #bayer_pad[\n> > + # border[0]:bayer_pad.shape[0] - border[0],\n> > + # border[1]:bayer_pad.shape[1] - border[1],\n> > + # :] = bayer\n> > + #bayer = bayer_pad\n> > +\n> > + # In numpy >=1.7.0 just use np.pad (version in Raspbian is 1.6.2 at the\n> > + # time of writing...)\n> > + #\n> > + rgb = np.pad(rgb, [\n> > + (border[0], border[0]),\n> > + (border[1], border[1]),\n> > + (0, 0),\n> > + ], 'constant')\n> > + bayer = np.pad(bayer, [\n> > + (border[0], border[0]),\n> > + (border[1], border[1]),\n> > + (0, 0),\n> > + ], 'constant')\n> > +\n> > + # For each plane in the RGB data, we use a nifty numpy trick\n> > + # (as_strided) to construct a view over the plane of 3x3 matrices. We do\n> > + # the same for the bayer array, then use Einstein summation on each\n> > + # (np.sum is simpler, but copies the data so it's slower), and divide\n> > + # the results to get our weighted average:\n> > +\n> > + for plane in range(3):\n> > + p = rgb[..., plane]\n> > + b = bayer[..., plane]\n> > + pview = as_strided(p, shape=(\n> > + p.shape[0] - borders[0],\n> > + p.shape[1] - borders[1]) + window, strides=p.strides * 2)\n> > + bview = as_strided(b, shape=(\n> > + b.shape[0] - borders[0],\n> > + b.shape[1] - borders[1]) + window, strides=b.strides * 2)\n> > + psum = np.einsum('ijkl->ij', pview)\n> > + bsum = np.einsum('ijkl->ij', bview)\n> > + output[..., plane] = psum // bsum\n> > +\n> > + return output\n> > +\n> > +\n> > +\n> > +\n> > +def to_rgb(fmt, size, data):\n> > + w = size[0]\n> > + h = size[1]\n> > +\n> > + if fmt == \"YUYV\":\n> > + # YUV422\n> > + yuyv = data.reshape((h, w // 2 * 4))\n> > +\n> > + # YUV444\n> > + yuv = np.empty((h, w, 3), dtype=np.uint8)\n> > + yuv[:, :, 0] = yuyv[:, 0::2] # Y\n> > + yuv[:, :, 1] = yuyv[:, 1::4].repeat(2, axis=1) # U\n> > + yuv[:, :, 2] = yuyv[:, 3::4].repeat(2, axis=1) # V\n> > +\n> > + m = np.array([\n> > + [ 1.0, 1.0, 1.0],\n> > + [-0.000007154783816076815, -0.3441331386566162, 1.7720025777816772],\n> > + [ 1.4019975662231445, -0.7141380310058594 , 0.00001542569043522235]\n> > + ])\n> > +\n> > + rgb = np.dot(yuv, m)\n> > + rgb[:, :, 0] -= 179.45477266423404\n> > + rgb[:, :, 1] += 135.45870971679688\n> > + rgb[:, :, 2] -= 226.8183044444304\n> > + rgb = rgb.astype(np.uint8)\n> > +\n> > + elif fmt == \"RGB888\":\n> > + rgb = data.reshape((h, w, 3))\n> > + rgb[:, :, [0, 1, 2]] = rgb[:, :, [2, 1, 0]]\n> > +\n> > + elif fmt == \"BGR888\":\n> > + rgb = data.reshape((h, w, 3))\n> > +\n> > + elif fmt in [\"ARGB8888\", \"XRGB8888\"]:\n> > + rgb = data.reshape((h, w, 4))\n> > + rgb = np.flip(rgb, axis=2)\n> > + # drop alpha component\n> > + rgb = np.delete(rgb, np.s_[0::4], axis=2)\n> > +\n> > + elif fmt.startswith(\"S\"):\n> > + bayer_pattern = fmt[1:5]\n> > + bitspp = int(fmt[5:])\n> > +\n> > + # TODO: shifting leaves the lowest bits 0\n> > + if bitspp == 8:\n> > + data = data.reshape((h, w))\n> > + data = data.astype(np.uint16) << 8\n> > + elif bitspp in [10, 12]:\n> > + data = data.view(np.uint16)\n> > + data = data.reshape((h, w))\n> > + data = data << (16 - bitspp)\n> > + else:\n> > + raise Exception(\"Bad bitspp:\" + str(bitspp))\n> > +\n> > + idx = bayer_pattern.find(\"R\")\n> > + assert(idx != -1)\n> > + r0 = (idx % 2, idx // 2)\n> > +\n> > + idx = bayer_pattern.find(\"G\")\n> > + assert(idx != -1)\n> > + g0 = (idx % 2, idx // 2)\n> > +\n> > + idx = bayer_pattern.find(\"G\", idx + 1)\n> > + assert(idx != -1)\n> > + g1 = (idx % 2, idx // 2)\n> > +\n> > + idx = bayer_pattern.find(\"B\")\n> > + assert(idx != -1)\n> > + b0 = (idx % 2, idx // 2)\n> > +\n> > + rgb = separate_components(data, r0, g0, g1, b0)\n> > + rgb = demosaic(rgb, r0, g0, g1, b0)\n> > + rgb = (rgb >> 8).astype(np.uint8)\n> > +\n> > + else:\n> > + rgb = None\n> > +\n> > + return rgb\n> > +\n> > +\n> > +class QtRenderer:\n> > + def __init__(self, state):\n> > + self.state = state\n> > +\n> > + self.cm = state[\"cm\"]\n> > + self.contexts = state[\"contexts\"]\n> > +\n> > + def setup(self):\n> > + self.app = QtWidgets.QApplication([])\n> > +\n> > + windows = []\n> > +\n> > + for ctx in self.contexts:\n> > + camera = ctx[\"camera\"]\n> > +\n> > + for stream in ctx[\"streams\"]:\n> > + fmt = stream.configuration.fmt\n> > + size = stream.configuration.size\n> > +\n> > + window = MainWindow(ctx, stream)\n> > + window.setAttribute(QtCore.Qt.WA_ShowWithoutActivating)\n> > + window.show()\n> > + windows.append(window)\n> > +\n> > + self.windows = windows\n> > +\n> > + def run(self):\n> > + camnotif = QtCore.QSocketNotifier(self.cm.efd, QtCore.QSocketNotifier.Read)\n> > + camnotif.activated.connect(lambda x: self.readcam())\n> > +\n> > + keynotif = QtCore.QSocketNotifier(sys.stdin.fileno(), QtCore.QSocketNotifier.Read)\n> > + keynotif.activated.connect(lambda x: self.readkey())\n> > +\n> > + print(\"Capturing...\")\n> > +\n> > + self.app.exec()\n> > +\n> > + print(\"Exiting...\")\n> > +\n> > + def readcam(self):\n> > + running = self.state[\"event_handler\"](self.state)\n> > +\n> > + if not running:\n> > + self.app.quit()\n> > +\n> > + def readkey(self):\n> > + sys.stdin.readline()\n> > + self.app.quit()\n> > +\n> > + def request_handler(self, ctx, req):\n> > + buffers = req.buffers\n> > +\n> > + for stream, fb in buffers.items():\n> > + wnd = next(wnd for wnd in self.windows if wnd.stream == stream)\n> > +\n> > + wnd.handle_request(stream, fb)\n> > +\n> > + self.state[\"request_prcessed\"](ctx, req)\n> > +\n> > + def cleanup(self):\n> > + for w in self.windows:\n> > + w.close()\n> > +\n> > +\n> > +class MainWindow(QtWidgets.QWidget):\n> > + def __init__(self, ctx, stream):\n> > + super().__init__()\n> > +\n> > + self.ctx = ctx\n> > + self.stream = stream\n> > +\n> > + self.label = QtWidgets.QLabel()\n> > +\n> > + windowLayout = QtWidgets.QHBoxLayout()\n> > + self.setLayout(windowLayout)\n> > +\n> > + windowLayout.addWidget(self.label)\n> > +\n> > + controlsLayout = QtWidgets.QVBoxLayout()\n> > + windowLayout.addLayout(controlsLayout)\n> > +\n> > + windowLayout.addStretch()\n> > +\n> > + group = QtWidgets.QGroupBox(\"Info\")\n> > + groupLayout = QtWidgets.QVBoxLayout()\n> > + group.setLayout(groupLayout)\n> > + controlsLayout.addWidget(group)\n> > +\n> > + lab = QtWidgets.QLabel(ctx[\"id\"])\n> > + groupLayout.addWidget(lab)\n> > +\n> > + self.frameLabel = QtWidgets.QLabel()\n> > + groupLayout.addWidget(self.frameLabel)\n> > +\n> > +\n> > + group = QtWidgets.QGroupBox(\"Properties\")\n> > + groupLayout = QtWidgets.QVBoxLayout()\n> > + group.setLayout(groupLayout)\n> > + controlsLayout.addWidget(group)\n> > +\n> > + camera = ctx[\"camera\"]\n> > +\n> > + for k, v in camera.properties.items():\n> > + lab = QtWidgets.QLabel()\n> > + lab.setText(k + \" = \" + str(v))\n> > + groupLayout.addWidget(lab)\n> > +\n> > + group = QtWidgets.QGroupBox(\"Controls\")\n> > + groupLayout = QtWidgets.QVBoxLayout()\n> > + group.setLayout(groupLayout)\n> > + controlsLayout.addWidget(group)\n> > +\n> > + for k, (min, max, default) in camera.controls.items():\n> > + lab = QtWidgets.QLabel()\n> > + lab.setText(\"{} = {}/{}/{}\".format(k, min, max, default))\n> > + groupLayout.addWidget(lab)\n> > +\n> > + controlsLayout.addStretch()\n> > +\n> > + def buf_to_qpixmap(self, stream, fb):\n> > + with fb.mmap(0) as b:\n> > + cfg = stream.configuration\n> > + w, h = cfg.size\n> > + pitch = cfg.stride\n> > +\n> > + if cfg.fmt == \"MJPEG\":\n> > + img = Image.open(BytesIO(b))\n> > + qim = ImageQt(img).copy()\n> > + pix = QtGui.QPixmap.fromImage(qim)\n> > + else:\n> > + data = np.array(b, dtype=np.uint8)\n> > + rgb = to_rgb(cfg.fmt, cfg.size, data)\n> > +\n> > + if rgb is None:\n> > + raise Exception(\"Format not supported: \" + cfg.fmt)\n> > +\n> > + pix = rgb_to_pix(rgb)\n> > +\n> > + return pix\n> > +\n> > + def handle_request(self, stream, fb):\n> > + ctx = self.ctx\n> > +\n> > + pix = self.buf_to_qpixmap(stream, fb)\n> > + self.label.setPixmap(pix)\n> > +\n> > + self.frameLabel.setText(\"Queued: {}\\nDone: {}\\nFps: {:.2f}\"\n> > + .format(ctx[\"reqs-queued\"], ctx[\"reqs-completed\"], ctx[\"fps\"]))\n> > diff --git a/src/py/test/cam_qtgl.py b/src/py/test/cam_qtgl.py\n> > new file mode 100644\n> > index 00000000..01168979\n> > --- /dev/null\n> > +++ b/src/py/test/cam_qtgl.py\n> > @@ -0,0 +1,385 @@\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +\n> > +from PyQt5 import QtCore, QtWidgets\n> > +from PyQt5.QtCore import Qt\n> > +\n> > +import numpy as np\n> > +import sys\n> > +import os\n> > +os.environ[\"PYOPENGL_PLATFORM\"] = \"egl\"\n> > +import math\n> > +\n> > +import OpenGL\n> > +#OpenGL.FULL_LOGGING = True\n> > +\n> > +from OpenGL import GL as gl\n> > +from OpenGL.EGL.KHR.image import *\n> > +from OpenGL.EGL.EXT.image_dma_buf_import import *\n> > +from OpenGL.EGL.VERSION.EGL_1_0 import *\n> > +from OpenGL.EGL.VERSION.EGL_1_2 import *\n> > +from OpenGL.EGL.VERSION.EGL_1_3 import *\n> > +\n> > +from OpenGL.GLES2.VERSION.GLES2_2_0 import *\n> > +from OpenGL.GLES2.OES.EGL_image import *\n> > +from OpenGL.GLES2.OES.EGL_image_external import *\n> > +from OpenGL.GLES3.VERSION.GLES3_3_0 import *\n> > +\n> > +from OpenGL.GL import shaders\n> > +\n> > +from gl_helpers import *\n> > +\n> > +# libcamera format string -> DRM fourcc\n> > +FMT_MAP = {\n> > + \"RGB888\": \"RG24\",\n> > + \"XRGB8888\": \"XR24\",\n> > + \"ARGB8888\": \"AR24\",\n> > + \"YUYV\": \"YUYV\",\n> > +}\n> > +\n> > +class EglState:\n> > + def __init__(self):\n> > + self.create_display()\n> > + self.choose_config()\n> > + self.create_context()\n> > + self.check_extensions()\n> > +\n> > + def create_display(self):\n> > + xdpy = getEGLNativeDisplay()\n> > + dpy = eglGetDisplay(xdpy)\n> > + self.display = dpy\n> > +\n> > + def choose_config(self):\n> > + dpy = self.display\n> > +\n> > + major, minor = EGLint(), EGLint()\n> > +\n> > + b = eglInitialize(dpy, major, minor)\n> > + assert(b)\n> > +\n> > + print(\"EGL {} {}\".format(\n> > + eglQueryString(dpy, EGL_VENDOR).decode(),\n> > + eglQueryString(dpy, EGL_VERSION).decode()))\n> > +\n> > + check_egl_extensions(dpy, [\"EGL_EXT_image_dma_buf_import\"])\n> > +\n> > + b = eglBindAPI(EGL_OPENGL_ES_API)\n> > + assert(b)\n> > +\n> > + def print_config(dpy, cfg):\n> > +\n> > + def _getconf(dpy, cfg, a):\n> > + value = ctypes.c_long()\n> > + eglGetConfigAttrib(dpy, cfg, a, value)\n> > + return value.value\n> > +\n> > + getconf = lambda a: _getconf(dpy, cfg, a)\n> > +\n> > + print(\"EGL Config {}: color buf {}/{}/{}/{} = {}, depth {}, stencil {}, native visualid {}, native visualtype {}\".format(\n> > + getconf(EGL_CONFIG_ID),\n> > + getconf(EGL_ALPHA_SIZE),\n> > + getconf(EGL_RED_SIZE),\n> > + getconf(EGL_GREEN_SIZE),\n> > + getconf(EGL_BLUE_SIZE),\n> > + getconf(EGL_BUFFER_SIZE),\n> > + getconf(EGL_DEPTH_SIZE),\n> > + getconf(EGL_STENCIL_SIZE),\n> > + getconf(EGL_NATIVE_VISUAL_ID),\n> > + getconf(EGL_NATIVE_VISUAL_TYPE)))\n> > +\n> > + if False:\n> > + num_configs = ctypes.c_long()\n> > + eglGetConfigs(dpy, None, 0, num_configs)\n> > + print(\"{} configs\".format(num_configs.value))\n> > +\n> > + configs = (EGLConfig * num_configs.value)()\n> > + eglGetConfigs(dpy, configs, num_configs.value, num_configs)\n> > + for config_id in configs:\n> > + print_config(dpy, config_id)\n> > +\n> > +\n> > + config_attribs = [\n> > + EGL_SURFACE_TYPE, EGL_WINDOW_BIT,\n> > + EGL_RED_SIZE, 8,\n> > + EGL_GREEN_SIZE, 8,\n> > + EGL_BLUE_SIZE, 8,\n> > + EGL_ALPHA_SIZE, 0,\n> > + EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT,\n> > + EGL_NONE,\n> > + ]\n> > +\n> > + n = EGLint()\n> > + configs = (EGLConfig * 1)()\n> > + b = eglChooseConfig(dpy, config_attribs, configs, 1, n)\n> > + assert(b and n.value == 1)\n> > + config = configs[0]\n> > +\n> > + print(\"Chosen Config:\")\n> > + print_config(dpy, config)\n> > +\n> > + self.config = config\n> > +\n> > + def create_context(self):\n> > + dpy = self.display\n> > +\n> > + context_attribs = [\n> > + EGL_CONTEXT_CLIENT_VERSION, 2,\n> > + EGL_NONE,\n> > + ]\n> > +\n> > + context = eglCreateContext(dpy, self.config, EGL_NO_CONTEXT, context_attribs)\n> > + assert(context)\n> > +\n> > + b = eglMakeCurrent(dpy, EGL_NO_SURFACE, EGL_NO_SURFACE, context)\n> > + assert(b)\n> > +\n> > + self.context = context\n> > +\n> > + def check_extensions(self):\n> > + check_gl_extensions([\"GL_OES_EGL_image\"])\n> > +\n> > + assert(eglCreateImageKHR)\n> > + assert(eglDestroyImageKHR)\n> > + assert(glEGLImageTargetTexture2DOES)\n> > +\n> > +\n> > +class QtRenderer:\n> > + def __init__(self, state):\n> > + self.state = state\n> > +\n> > + def setup(self):\n> > + self.app = QtWidgets.QApplication([])\n> > +\n> > + window = MainWindow(self.state)\n> > + window.setAttribute(QtCore.Qt.WA_ShowWithoutActivating)\n> > + window.show()\n> > +\n> > + self.window = window\n> > +\n> > + def run(self):\n> > + camnotif = QtCore.QSocketNotifier(self.state[\"cm\"].efd, QtCore.QSocketNotifier.Read)\n> > + camnotif.activated.connect(lambda x: self.readcam())\n> > +\n> > + keynotif = QtCore.QSocketNotifier(sys.stdin.fileno(), QtCore.QSocketNotifier.Read)\n> > + keynotif.activated.connect(lambda x: self.readkey())\n> > +\n> > + print(\"Capturing...\")\n> > +\n> > + self.app.exec()\n> > +\n> > + print(\"Exiting...\")\n> > +\n> > + def readcam(self):\n> > + running = self.state[\"event_handler\"](self.state)\n> > +\n> > + if not running:\n> > + self.app.quit()\n> > +\n> > + def readkey(self):\n> > + sys.stdin.readline()\n> > + self.app.quit()\n> > +\n> > + def request_handler(self, ctx, req):\n> > + self.window.handle_request(ctx, req)\n> > +\n> > + def cleanup(self):\n> > + self.window.close()\n> > +\n> > +\n> > +class MainWindow(QtWidgets.QWidget):\n> > + def __init__(self, state):\n> > + super().__init__()\n> > +\n> > + self.setAttribute(Qt.WA_PaintOnScreen)\n> > + self.setAttribute(Qt.WA_NativeWindow)\n> > +\n> > + self.state = state\n> > +\n> > + self.textures = {}\n> > + self.reqqueue = {}\n> > + self.current = {}\n> > +\n> > + for ctx in self.state[\"contexts\"]:\n> > +\n> > + self.reqqueue[ctx[\"idx\"]] = []\n> > + self.current[ctx[\"idx\"]] = []\n> > +\n> > + for stream in ctx[\"streams\"]:\n> > + fmt = stream.configuration.fmt\n> > + size = stream.configuration.size\n> > +\n> > + if not fmt in FMT_MAP:\n> > + raise Exception(\"Unsupported pixel format: \" + str(fmt))\n> > +\n> > + self.textures[stream] = None\n> > +\n> > + num_tiles = len(self.textures)\n> > + self.num_columns = math.ceil(math.sqrt(num_tiles))\n> > + self.num_rows = math.ceil(num_tiles / self.num_columns)\n> > +\n> > + self.egl = EglState()\n> > +\n> > + self.surface = None\n> > +\n> > + def paintEngine(self):\n> > + return None\n> > +\n> > + def create_surface(self):\n> > + native_surface = c_void_p(self.winId().__int__())\n> > + surface = eglCreateWindowSurface(self.egl.display, self.egl.config,\n> > + native_surface, None)\n> > +\n> > + b = eglMakeCurrent(self.egl.display, self.surface, self.surface, self.egl.context)\n> > + assert(b)\n> > +\n> > + self.surface = surface\n> > +\n> > + def init_gl(self):\n> > + self.create_surface()\n> > +\n> > + vertShaderSrc = \"\"\"\n> > + attribute vec2 aPosition;\n> > + varying vec2 texcoord;\n> > +\n> > + void main()\n> > + {\n> > + gl_Position = vec4(aPosition * 2.0 - 1.0, 0.0, 1.0);\n> > + texcoord.x = aPosition.x;\n> > + texcoord.y = 1.0 - aPosition.y;\n> > + }\n> > + \"\"\"\n> > + fragShaderSrc = \"\"\"\n> > + #extension GL_OES_EGL_image_external : enable\n> > + precision mediump float;\n> > + varying vec2 texcoord;\n> > + uniform samplerExternalOES texture;\n> > +\n> > + void main()\n> > + {\n> > + gl_FragColor = texture2D(texture, texcoord);\n> > + }\n> > + \"\"\"\n> > +\n> > + program = shaders.compileProgram(\n> > + shaders.compileShader(vertShaderSrc, GL_VERTEX_SHADER),\n> > + shaders.compileShader(fragShaderSrc, GL_FRAGMENT_SHADER)\n> > + )\n> > +\n> > + glUseProgram(program)\n> > +\n> > + glClearColor(0.5, 0.8, 0.7, 1.0)\n> > +\n> > + vertPositions = [\n> > + 0.0, 0.0,\n> > + 1.0, 0.0,\n> > + 1.0, 1.0,\n> > + 0.0, 1.0\n> > + ]\n> > +\n> > + inputAttrib = glGetAttribLocation(program, \"aPosition\")\n> > + glVertexAttribPointer(inputAttrib, 2, GL_FLOAT, GL_FALSE, 0, vertPositions)\n> > + glEnableVertexAttribArray(inputAttrib)\n> > +\n> > +\n> > + def create_texture(self, stream, fb):\n> > + cfg = stream.configuration\n> > + fmt = cfg.fmt\n> > + fmt = str_to_fourcc(FMT_MAP[fmt])\n> > + w, h = cfg.size\n> > +\n> > + attribs = [\n> > + EGL_WIDTH, w,\n> > + EGL_HEIGHT, h,\n> > + EGL_LINUX_DRM_FOURCC_EXT, fmt,\n> > + EGL_DMA_BUF_PLANE0_FD_EXT, fb.fd(0),\n> > + EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,\n> > + EGL_DMA_BUF_PLANE0_PITCH_EXT, cfg.stride,\n> > + EGL_NONE,\n> > + ]\n> > +\n> > + image = eglCreateImageKHR(self.egl.display,\n> > + EGL_NO_CONTEXT,\n> > + EGL_LINUX_DMA_BUF_EXT,\n> > + None,\n> > + attribs)\n\nWill be interesting to add dmabuf support to qcam at some point.\n\n> > + assert(image)\n> > +\n> > + textures = glGenTextures(1)\n> > + glBindTexture(GL_TEXTURE_EXTERNAL_OES, textures)\n> > + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR)\n> > + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR)\n> > + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)\n> > + glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)\n> > + glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, image)\n> > +\n> > + return textures\n> > +\n> > + def resizeEvent(self, event):\n> > + size = event.size()\n> > +\n> > + print(\"Resize\", size)\n> > +\n> > + super().resizeEvent(event)\n> > +\n> > + if self.surface == None:\n> > + return\n> > +\n> > + glViewport(0, 0, size.width()//2, size.height())\n> > +\n> > + def paintEvent(self, event):\n> > + if self.surface == None:\n> > + self.init_gl()\n> > +\n> > + for ctx_idx, queue in self.reqqueue.items():\n> > + if len(queue) == 0:\n> > + continue\n> > +\n> > + ctx = next(ctx for ctx in self.state[\"contexts\"] if ctx[\"idx\"] == ctx_idx)\n> > +\n> > + if self.current[ctx_idx]:\n> > + old = self.current[ctx_idx]\n> > + self.current[ctx_idx] = None\n> > + self.state[\"request_prcessed\"](ctx, old)\n> > +\n> > + next_req = queue.pop(0)\n> > + self.current[ctx_idx] = next_req\n> > +\n> > + stream, fb = next(iter(next_req.buffers.items()))\n> > +\n> > + self.textures[stream] = self.create_texture(stream, fb)\n> > +\n> > + self.paint_gl()\n> > +\n> > + def paint_gl(self):\n> > + b = eglMakeCurrent(self.egl.display, self.surface, self.surface, self.egl.context)\n> > + assert(b)\n> > +\n> > + glClear(GL_COLOR_BUFFER_BIT)\n> > +\n> > + size = self.size()\n> > +\n> > + for idx,ctx in enumerate(self.state[\"contexts\"]):\n> > + for stream in ctx[\"streams\"]:\n> > + if self.textures[stream] == None:\n> > + continue\n> > +\n> > + w = size.width() // self.num_columns\n> > + h = size.height() // self.num_rows\n> > +\n> > + x = idx % self.num_columns\n> > + y = idx // self.num_columns\n> > +\n> > + x *= w\n> > + y *= h\n> > +\n> > + glViewport(x, y, w, h)\n> > +\n> > + glBindTexture(GL_TEXTURE_EXTERNAL_OES, self.textures[stream])\n> > + glDrawArrays(GL_TRIANGLE_FAN, 0, 4)\n> > +\n> > + b = eglSwapBuffers(self.egl.display, self.surface)\n> > + assert(b)\n> > +\n> > + def handle_request(self, ctx, req):\n> > + self.reqqueue[ctx[\"idx\"]].append(req)\n> > + self.update()\n> > diff --git a/src/py/test/gl_helpers.py b/src/py/test/gl_helpers.py\n> > new file mode 100644\n> > index 00000000..a80b03b2\n> > --- /dev/null\n> > +++ b/src/py/test/gl_helpers.py\n> > @@ -0,0 +1,67 @@\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2021, Tomi Valkeinen \n> > +\n> > +from OpenGL.EGL.VERSION.EGL_1_0 import EGLNativeDisplayType, eglGetProcAddress, eglQueryString, EGL_EXTENSIONS\n> > +\n> > +from OpenGL.raw.GLES2 import _types as _cs\n> > +from OpenGL.GLES2.VERSION.GLES2_2_0 import *\n> > +from OpenGL.GLES3.VERSION.GLES3_3_0 import *\n> > +from OpenGL import GL as gl\n> > +\n> > +from ctypes import c_int, c_char_p, c_void_p, cdll, POINTER, util, \\\n> > + pointer, CFUNCTYPE, c_bool\n> > +\n> > +def getEGLNativeDisplay():\n> > + _x11lib = cdll.LoadLibrary(util.find_library(\"X11\"))\n> > + XOpenDisplay = _x11lib.XOpenDisplay\n> > + XOpenDisplay.argtypes = [c_char_p]\n> > + XOpenDisplay.restype = POINTER(EGLNativeDisplayType)\n> > +\n> > + xdpy = XOpenDisplay(None)\n> > +\n> > +# Hack. PyOpenGL doesn't seem to manage to find glEGLImageTargetTexture2DOES.\n> > +def getglEGLImageTargetTexture2DOES():\n> > + funcptr = eglGetProcAddress(\"glEGLImageTargetTexture2DOES\")\n> > + prototype = CFUNCTYPE(None,_cs.GLenum,_cs.GLeglImageOES)\n> > + return prototype(funcptr)\n> > +\n> > +glEGLImageTargetTexture2DOES = getglEGLImageTargetTexture2DOES()\n> > +\n> > +\n> > +def str_to_fourcc(str):\n> > + assert(len(str) == 4)\n> > + fourcc = 0\n> > + for i,v in enumerate([ord(c) for c in str]):\n> > + fourcc |= v << (i * 8)\n> > + return fourcc\n> > +\n> > +def get_gl_extensions():\n> > + n = GLint()\n> > + glGetIntegerv(GL_NUM_EXTENSIONS, n)\n> > + gl_extensions = []\n> > + for i in range(n.value):\n> > + gl_extensions.append(gl.glGetStringi(GL_EXTENSIONS, i).decode())\n> > + return gl_extensions\n> > +\n> > +def check_gl_extensions(required_extensions):\n> > + extensions = get_gl_extensions()\n> > +\n> > + if False:\n> > + print(\"GL EXTENSIONS: \", \" \".join(extensions))\n> > +\n> > + for ext in required_extensions:\n> > + if not ext in extensions:\n> > + raise Exception(ext + \" missing\")\n> > +\n> > +def get_egl_extensions(egl_display):\n> > + return eglQueryString(egl_display, EGL_EXTENSIONS).decode().split(\" \")\n> > +\n> > +def check_egl_extensions(egl_display, required_extensions):\n> > + extensions = get_egl_extensions(egl_display)\n> > +\n> > + if False:\n> > + print(\"EGL EXTENSIONS: \", \" \".join(extensions))\n> > +\n> > + for ext in required_extensions:\n> > + if not ext in extensions:\n> > + raise Exception(ext + \" missing\")","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 558DCBDB13\n\tfor ;\n\tThu, 9 Dec 2021 19:44:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8FE4E60882;\n\tThu, 9 Dec 2021 20:44:44 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2BE91607DE\n\tfor ;\n\tThu, 9 Dec 2021 20:44:43 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6B754501;\n\tThu, 9 Dec 2021 20:44:42 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"px0sv9QE\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1639079082;\n\tbh=wQOIVvRjfzCN0pzClUQ/SRKKRPJeCcbAykazDqAdhzM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=px0sv9QEtChaKIs5Mh1SF3RkdfdE3loEm/fFKdDCD1bkVRaviqgwpqCYXUQJRdnLB\n\t1yh+gKxGZkGLyQdZItYFuhh619Jan4Ilb52G5m5GFN6gfl0GzglorY9shJ7JQYUQJ6\n\tzH3iW/O5rRFEBzbvD6Tx20alwZuOA9qrlLVa/8oQ=","Date":"Thu, 9 Dec 2021 21:44:13 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"","References":"<20211209092906.37303-1-tomi.valkeinen@ideasonboard.com>\n\t<20211209092906.37303-6-tomi.valkeinen@ideasonboard.com>\n\t<163904421518.2066819.4960395426215054722@Monstersaurus>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<163904421518.2066819.4960395426215054722@Monstersaurus>","Subject":"Re: [libcamera-devel] [RFC v3 5/5] py: Add cam.py","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]