{"id":17597,"url":"https://patchwork.libcamera.org/api/covers/17597/?format=json","web_url":"https://patchwork.libcamera.org/cover/17597/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20221014131846.27169-1-naush@raspberrypi.com>","date":"2022-10-14T13:18:36","name":"[libcamera-devel,v1,00/10] Raspberry Pi: Platform configuration and buffer allocation improvements","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"mbox":"https://patchwork.libcamera.org/cover/17597/mbox/","series":[{"id":3555,"url":"https://patchwork.libcamera.org/api/series/3555/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3555","date":"2022-10-14T13:18:37","name":"Raspberry Pi: Platform configuration and buffer allocation improvements","version":1,"mbox":"https://patchwork.libcamera.org/series/3555/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/17597/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B4370C327C\n\tfor ;\n\tFri, 14 Oct 2022 13:18:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2926462DC4;\n\tFri, 14 Oct 2022 15:18:56 +0200 (CEST)","from mail-wr1-x435.google.com (mail-wr1-x435.google.com\n\t[IPv6:2a00:1450:4864:20::435])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2434562D8E\n\tfor ;\n\tFri, 14 Oct 2022 15:18:51 +0200 (CEST)","by mail-wr1-x435.google.com with SMTP id r13so7492229wrj.11\n\tfor ;\n\tFri, 14 Oct 2022 06:18:51 -0700 (PDT)","from naush-laptop.localdomain ([93.93.133.154])\n\tby smtp.gmail.com with ESMTPSA id\n\tg17-20020a05600c001100b003c6bbe910fdsm8282458wmc.9.2022.10.14.06.18.48\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 14 Oct 2022 06:18:49 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1665753536;\n\tbh=ExRPpGRFqjnrmv9X2kH7z0Ok9SOOjLAPX+Olr3TDYnQ=;\n\th=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post:\n\tList-Help:List-Subscribe:From:Reply-To:From;\n\tb=JaBFlN7n7EMeka8PFxhKQzUJbAohfd2dxJTp7yihhGVJjgTblZb1iPzDCdCfIbb4O\n\tdHWjt6NMa695BwqOjxHeOk41tSWPUZ+ePOWGLcHJPkQfx7l88KC07cm+NGc7juiKBy\n\tKZ+0CjERQcWFaWVLjVVSZCAL1ACjSFpeO3j9hR7nhrXqoEPkFsjeJ4XGsv+fE58ijw\n\tpCfAmq7cPp349AWRfTf4RCGDCfzgHMMpQ2lHkv8vRHWnWbSQa0ZzgIW3CRB++WA+3u\n\tuB9g+8ztLr7KExnkK6U8xUsED5APfQBcbd9resLOip8OpUb/RYKpbDpjS/ynA8AQ/X\n\tKZbqqjwy8SVuA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:from:to:cc:subject:date:message-id:reply-to;\n\tbh=L+1yEG0ZioKF335bKGvQvDrhQdwlXiacOlC6EMfirlU=;\n\tb=WUCruMWcTV9xIRGiSlphwQKDU7Ed5ZQj1TEnIQCknMBlY894XWkqdMTqNJt6ZpYufG\n\tiM6M02ZjvZp7lhBF8Ef7rb5R499B/EerzgpXhXEpbHMnBXPAWa7RO27tA4c+S+1nXO6B\n\tg0GKloRpTa0AqT+QRz+Oie3JQhY43H80pL8nf4LVe+YMbuBAUGKCvvSd/Pzvl9efalTJ\n\tetICXOpVvMaFOl3SykwyOOH/KP0MidQMGupyJnWs8osV3z0ocMjwqluLh6U8X5NACSFw\n\tpXERnTqNarUVbihDCA+ZxiHjpOPZWJdk+VagcKT9Yx0x9cXcbDAozsv/Ir1iEXERtpxZ\n\tXOBQ=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"WUCruMWc\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=L+1yEG0ZioKF335bKGvQvDrhQdwlXiacOlC6EMfirlU=;\n\tb=E9R92UGOiDdPmD4sdmfU8T5e03rTb3Ft/5J8RVk80z3vGhCrN6RCVGGOunqna8a0fV\n\tg9NDbUgjq9Th/wf4whpgkxvaEpGWzrXXkOyiOwPY2+wo8/tY8+Ps7/AquIGBf/4MqplX\n\tzJmRqw5U7H81hYkilKoNCKgEADfoL/iPdUQdnt4Clw/mj2v1wh0nGPsxywlpN+1ao/Sz\n\taQOE6EPJoy5W1aeZslzqj2KZ6FRF/IhNzbXgi6n9y0kdSTN0DaXhf7qCXsSgsYTpuwjd\n\tlxK8k2rodHg0l3i2BWnMoOgDpADItizsFlF1ta0CeJ0BKFf8YPVxKuqI5tMipzgXE5xp\n\tiDzQ==","X-Gm-Message-State":"ACrzQf0ajeAXf95H8e73duDAP9MlI4l2BKKRH8kXhAe2Iv5BsUResHX2\n\t0RkU+u2XPjWmORTOf+fGf+hapHGNYTy8oA==","X-Google-Smtp-Source":"AMsMyM7828FME1Kj1Ye7L5pDhnp6j5RrAsVFGkBo6OZukj5kKBPVoZxGZDcFka3VFPmpK5M0S2O+Ug==","X-Received":"by 2002:a05:6000:1d8a:b0:22f:2235:93cb with SMTP id\n\tbk10-20020a0560001d8a00b0022f223593cbmr3357775wrb.701.1665753530204; \n\tFri, 14 Oct 2022 06:18:50 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 14 Oct 2022 14:18:36 +0100","Message-Id":"<20221014131846.27169-1-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.25.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v1 00/10] Raspberry Pi: Platform\n\tconfiguration and buffer allocation improvements","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Naushir Patuck via libcamera-devel\n\t","Reply-To":"Naushir Patuck ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi,\n\nOne big issue that is plaguing libcamera users on Raspberry Pi is memory usage.\nAll our frame buffers have to be allocated in CMA space because of hardware\nrestrictions. Unfortunately CMA space is very limited (particularly on the\nRaspberry Pi Zero), and is prone to frequent fragmentation as display/rendering\nbuffers also use this pool. Not to mention that the kernel is also allowed to\nallocate in the CMA pool for its own needs. As a result, some of our libcamera\napplications intermittently fail to run because of buffer allocation failure,\nand some use-cases (e.g. > 20MPix captures) will simply not be able to run on\nsome platforms at all.\n\nThis patch series attempts to try and fix this by easing the memory usage of\nthe Raspberry Pi pipeline handler. It does this by using a platform configuration\nfile which (amongst other things) instructs the pipeline handler on how many\ninternal buffers it should allocate. There may be a performance penality (e.g\ndropped frames, or longer time taken for initial AE/AWB convergence) that have\nbeen documented. But if the application can guarantee certain behavior, we can\nensure the pipeline can still process requests with the same performance as we\nhave today. The Raspberry Pi libcamera-apps and picamera2 framework already use\nthis behavior, so the change should be transparent to applications built on\nthese, apart from using significantly less memory.\n\nPatches 1 to 4 and the framework for using a configuration file to setup the\nbuffer allocation parameters\n\nPatch 5 to 7 improve the buffer handling dealing with dropped frames, allowing\nus to use fewer internal buffers.\n\nPatch 8 adds a parameter to modify drop frame behavior - not related to this\nproblem, but a nice to have.\n\nPatches 9 and 10 add configuration files for minimal memory configuration, and\nanother for optimal memory usage for libcamera-apps and picamera2.\n\nI've considered how these various performance tuning parameters could be made\npart of the core StreamConfiguration structure or even as a Control. However,\nthey are entirely platform specific, and I am struggling to come up with a clean\nway to put this into libcamera core with polluting it with various platform specific\nparameters. If others have any ideas or suggestions on using alternate means\nof passing platform specific parameters into the pipeline handler, please do\ncomment on this review.\n\nThanks,\nNaush\n\nNaushir Patuck (10):\n libcamera: pipeline: Add a platform configuration file helper\n pipeline: raspberrypi: Add a pipeline config structure\n pipeline: raspberrypi: Split out ISP Output0 buffer allocation\n pipeline: raspberrypi: Read config parameters from a file\n pipeline: raspberrypi: Disable StreamOn for ISP Output0/1 when\n dropping frames\n pipeline: raspberrypi: Reorder startup drop frame initialisation\n pipeline: raspberrypi: Add a parameter to disable startup drop frames\n pipeline: raspberrypi: Allow pipeline handler to always use the newest\n frame\n pipeline: raspberrypi: Add minimal memory usage config file\n pipeline: raspberrypi: Add libcamera-apps and picamera2 config file\n\n include/libcamera/internal/pipeline_handler.h | 2 +\n .../pipeline/raspberrypi/data/default.json | 28 +++\n .../pipeline/raspberrypi/data/meson.build | 10 +\n .../raspberrypi/data/minimal_mem.json | 28 +++\n .../pipeline/raspberrypi/data/rpi_apps.json | 28 +++\n .../pipeline/raspberrypi/meson.build | 2 +\n .../pipeline/raspberrypi/raspberrypi.cpp | 215 +++++++++++++++---\n src/libcamera/pipeline_handler.cpp | 57 +++++\n 8 files changed, 339 insertions(+), 31 deletions(-)\n create mode 100644 src/libcamera/pipeline/raspberrypi/data/default.json\n create mode 100644 src/libcamera/pipeline/raspberrypi/data/meson.build\n create mode 100644 src/libcamera/pipeline/raspberrypi/data/minimal_mem.json\n create mode 100644 src/libcamera/pipeline/raspberrypi/data/rpi_apps.json"}