From patchwork Fri Nov 12 10:03:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14587 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 90657BF415 for ; Fri, 12 Nov 2021 10:03:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 46C3D6035D; Fri, 12 Nov 2021 11:03:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="LStabVOo"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E0E960234 for ; Fri, 12 Nov 2021 11:03:47 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id y196so7213624wmc.3 for ; Fri, 12 Nov 2021 02:03:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cXVNeGLXDz6JtvQxKGu1TUTRWymUo0eSitm6CGOcfkI=; b=LStabVOozJlLjZrzCf04cYr9hNSg975Ni4o/2Oe5TgSOiTnsWlHjpyx0hY0oe1k1u9 ReMYRDxYp6Qa2cFgfhzFjYn/B443CIaronX6j+EXSa8l0oIYk5kLgEG+qTlS+lc16hUs ADTg+WY22vy21DXwSLLYevL8sGv3JtHL+Dw29w2O4+yoxd57A3R4dPr6/dckJ4C/hyW7 Y/OmbZgihmTtzPVeuqPqPdr6I2zT+PvUVLxYO6Am7Q6GpE3z5rz7O0V0IhrxcwEkcDWY fc7ttOBf4qJvRpXLIVfN1OG44D9GxIBT4Mw39G3/hB9eQU+gOnm6SFsHXv9ll0EvyJmi Gmzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cXVNeGLXDz6JtvQxKGu1TUTRWymUo0eSitm6CGOcfkI=; b=SDO6XCjDcdW4w4Vr/Sk+OL4xm/lysva5dOM+ErHWepiSybxq8RcNXk7nXe5qslBf1B +CYAIokzvB/Hv1XaieROyEdfhgKFI8GwGDGOItgSiF+OmljAKwPuJktfdyZaSdywNr5a 7laJoTQx1oF3b4LDe97pbVjU3LBvpTHKTUk7Lac2bolEm2nYmxEfRSVNC2m7MjIpKpG9 qyRAog2aIHhW61IeO8lKlx34ijwW4aWse7ui0iPY4fFPLk8Adk5Ubbr7eVLP0fvLpxOH ITp5O5QoxJt2i22rUKXI7g/BxnyxlKws6abkSBwIpEvamOGWRrDP1tdRjVQcuvPZ8fDE EYKQ== X-Gm-Message-State: AOAM532YncQ1uLD7Ndkif1aOUa+QQXtqCZbKDfJHZsmPGAJ/Wcv1dF41 2yD5LxGF+z5zQVPQHdqCHxNgxGxWeiAjBBkr X-Google-Smtp-Source: ABdhPJzAcZEaLvGjN+3bXe7ONYWJhW0lJ9/eEHE7PqpWgmQHPqtTpOgsgQcFoRt89y4DqJqN/Dm93A== X-Received: by 2002:a05:600c:2297:: with SMTP id 23mr32152915wmf.73.1636711426629; Fri, 12 Nov 2021 02:03:46 -0800 (PST) Received: from naush-laptop.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id o9sm5283155wrs.4.2021.11.12.02.03.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 02:03:46 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 12 Nov 2021 10:03:03 +0000 Message-Id: <20211112100305.2217099-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211112100305.2217099-1-naush@raspberrypi.com> References: <20211112100305.2217099-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] pipeline: raspberrypi: Add const qualifer in isRaw() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This function does not modify the pixFmt parameter, so use a const qualifier. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f227398a9..11d3c2b120dd 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,7 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, return deviceFormat; } -bool isRaw(PixelFormat &pixFmt) +bool isRaw(const PixelFormat &pixFmt) { /* * The isRaw test might be redundant right now the pipeline handler only From patchwork Fri Nov 12 10:03:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14588 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 00F40BF415 for ; Fri, 12 Nov 2021 10:03:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0B4B60362; Fri, 12 Nov 2021 11:03:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lkk8ZtOs"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7AA6060234 for ; Fri, 12 Nov 2021 11:03:51 +0100 (CET) Received: by mail-wr1-x429.google.com with SMTP id n29so14477840wra.11 for ; Fri, 12 Nov 2021 02:03:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P0NCeHwS7qg46ZoOFp+cM5MU1LaBQjc7ALSHiOuZ6cs=; b=lkk8ZtOsWFlyed0y5zMEU4v43p4KPuZAX+ovlMcnQM0yqsF7HKsYuHgtU1ZHDfwtLM ypxUyAbcnvnxGq31kPh70NGkY08gorEm2qHtFurDl3oBqdKAevvH/Dac/DDfVOL7kwIG VDeoOU58LOiC0G1KuVHocVGsw2JJDC574/UYPIOuDQDXlXhr6jTZqsezUydwOtFL1L/C uIwiPPPh862fkRYQspJhPpVmqtqtejUyEO927IAgn4ldPTcuwbDzzIWIJOKxOzfJDIZR ege5h5D8siA5jyZvzcFgUfaRPCbe+8G49PiNdieNcKS+vI3h/Ycj8Q4kdbQjwcTSmjEN up5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P0NCeHwS7qg46ZoOFp+cM5MU1LaBQjc7ALSHiOuZ6cs=; b=v5oVqL1i0aC5Ikkz0lbh+nO+TgT57hPXd/q2HudwPWr9kcI8M8XopsY+M/m16cWlSP uqq4pqpimahR/qdIXzmyydptAwsX714sgxY8hm7XuRt0ppuBIeJYDK230yxeBAtJU853 EcV74Eo8m138b+KFYGTdiE+ssFsQZTHBY+jcynvlZrAOjfK9MQHBlokoC3eaAyvriOxb ohFgL7lGp2G0mbecuXPrRAOCwoUjT37nuQSHy6C1v0BbdSPFOiDTC/fIbQVucNAt1Rf3 EX+AvT/C/N/e2QWI4UPkOvNsZb5Va0KZw85xabjqP+lPh8XsUusVBgWedkfmQE2B/7GZ Vz8g== X-Gm-Message-State: AOAM532VreweANCI1AQpChmSON6JMG/akzeb+18Xl0byP3uEyctOzV5k lnAVQTovEVdzJ32WAW3J+LnUlCiTEmZrv4Fo X-Google-Smtp-Source: ABdhPJzDRXbhTyBpQFbfuTyqkWwC0nnDPr/sTstNgBk3/2lj/s7MOa66nDzr1sUqVqRaakNZuAwMrg== X-Received: by 2002:a5d:6c6b:: with SMTP id r11mr17054706wrz.231.1636711430994; Fri, 12 Nov 2021 02:03:50 -0800 (PST) Received: from naush-laptop.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id o9sm5283155wrs.4.2021.11.12.02.03.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 02:03:50 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 12 Nov 2021 10:03:04 +0000 Message-Id: <20211112100305.2217099-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211112100305.2217099-1-naush@raspberrypi.com> References: <20211112100305.2217099-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] pipeline: raspberrypi: Rework the internal buffer allocation scheme X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For simplicity, the pipeline handler currently look at the maximum number of buffers set in the StreamConfiguration by the user and allocate the same number of internal buffers for all device nodes. This would likely overallocate buffers for some nodes. Rework this logic to try and minimise overallcations without compromising performance. The key change is to mostly decouple the number of internal buffers allocated from number of buffers requested by the user through the StreamConfiguration. For ISP nodes, we only ever need 1 set of internal buffers, as the hardware runs synchronous with the requests and IPA. For Unicam nodes, allocate a minimum for 4 buffers (exported + internal), but also require at least 2 internal buffers to minimise frame drops. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 43 ++++++++++++++----- 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 11d3c2b120dd..4f6c699a4379 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1211,21 +1211,42 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); + unsigned int numRawBuffers = 0; int ret; - /* - * Decide how many internal buffers to allocate. For now, simply look - * at how many external buffers will be provided. We'll need to improve - * this logic. However, we really must have all streams allocate the same - * number of buffers to simplify error handling in queueRequestDevice(). - */ - unsigned int maxBuffers = 0; - for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) - maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); + for (Stream *s : camera->streams()) { + if (isRaw(s->configuration().pixelFormat)) { + numRawBuffers = s->configuration().bufferCount; + break; + } + } + /* Decide how many internal buffers to allocate. */ for (auto const stream : data->streams_) { - ret = stream->prepareBuffers(maxBuffers); + unsigned int numBuffers; + + if (stream == &data->unicam_[Unicam::Image] || + stream == &data->unicam_[Unicam::Embedded]) { + /* + * For Unicam, allocate a minimum of 4 buffers as we want + * to avoid any frame drops. If an application has configured + * a RAW stream, allocate additional buffers to make up the + * minimum, but ensure we have at least 2 sets of internal + * buffers to use to minimise frame drops. + */ + constexpr unsigned int minBuffers = 4; + numBuffers = std::max(2, minBuffers - numRawBuffers); + } else { + /* + * Since the ISP runs synchronous with the IPA and requests, + * we only ever need one set of internal buffers. Any buffers + * the application wants to hold onto will already be exported + * through PipelineHandlerRPi::exportFrameBuffers(). + */ + numBuffers = 1; + } + + ret = stream->prepareBuffers(numBuffers); if (ret < 0) return ret; } From patchwork Fri Nov 12 10:03:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14589 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CE636BF415 for ; Fri, 12 Nov 2021 10:03:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7E2CE60369; Fri, 12 Nov 2021 11:03:55 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AMMhRK7p"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F6CB60234 for ; Fri, 12 Nov 2021 11:03:54 +0100 (CET) Received: by mail-wr1-x42a.google.com with SMTP id n29so14478132wra.11 for ; Fri, 12 Nov 2021 02:03:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3DsHUzto1xICrV+8oKQbV2fo+5EHRpl8Rwp/bFHynbY=; b=AMMhRK7p/PTuQPBjoY7S0Qal3ebR8jiz7ZS1zJcskOKTydC4aiEA6kH//ZtemWm+4N PvvYuy3izapBO2i0HQL1UXrmPtDad78q6PCgm8ldHdgCgVJAQuaEIHmM8utEM6SNE6pc 3UXijfrBWIENNNZC3RKbz8oS+7rDMZRX1/20DKTCKB23cQzp+l97apbsrJ0x3DYKHBZb sxdXSRsncUs1Tm0rp/13oR4W3RWc8IcedJR+IbUFBWoHjujgvQ1gPmmEdKblfxMdBF3/ 6GFpxrkmaxrFqL5DxbRkEJ4uySubDzgF9F1HQ8jnoITJxGV/4AMqrWhym5NVaEfsLql1 5Jxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3DsHUzto1xICrV+8oKQbV2fo+5EHRpl8Rwp/bFHynbY=; b=dsXyEwprUym7wf8pYsRLgzlnS5iqy5eKgroKuPGayXeu7XQO5CScZiz5R61MLLthlU 6znw7pwSsBpPbqg8TU+cd0KB8wcF2iXJwnqsgLdBt7HdZXgKmiNPMSHmQpfVWYL7yfqJ hTKfba2JHMtGN7VFyx3iY8WUylBq+yY81wcqVwwXvF58m0rQhw29iNWz8a85oz+NFR38 rEQ8JRTajSrxeZh+7lddJhbUSrQAa15OURaZMIQSjc2GuHzsiH+x8WKe6t7BUXOXbJiv 32xmsLNlqbxaR1EyHBR7lVpATHBgP5LQgQR7YK8DJM1n/aFL09SjF3LaSzAq7voHazT+ 62Ng== X-Gm-Message-State: AOAM533fIelrMmvEG5xLxOmZDc50wk2t48fnOqmpeQP17oGxIwo7zat5 kPFIhU8fZefU+SoSPOw9xWoR6/pe4DD3wPQh X-Google-Smtp-Source: ABdhPJzgmvxamQD3/yqa+Imafnc1ApHcdE10N3WlWK9e5TB7KEkJIxg5Lf868TOEAeizT3sDDsTanQ== X-Received: by 2002:adf:d1e2:: with SMTP id g2mr16599685wrd.179.1636711433839; Fri, 12 Nov 2021 02:03:53 -0800 (PST) Received: from naush-laptop.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id o9sm5283155wrs.4.2021.11.12.02.03.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 02:03:53 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 12 Nov 2021 10:03:05 +0000 Message-Id: <20211112100305.2217099-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211112100305.2217099-1-naush@raspberrypi.com> References: <20211112100305.2217099-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] pipeline: raspberrypi: Increase the V4L2BufferCache slot allocations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If a stream is marked as external, double the number of V4L2BufferCache slots that are allocated. This is to account for additional buffers that may be allocated directly by the application. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/raspberrypi/rpi_stream.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b3265d0e8aab..67901936d6b6 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -107,6 +107,14 @@ int Stream::prepareBuffers(unsigned int count) count = bufferMap_.size(); } + /* + * If this is an external stream, we must allocate slots for buffers that + * might be externally allocated. We have no indication of how many buffers + * may be used, so this might overallocate slots in the buffer cache. + */ + if (isExternal()) + count = count * 2; + return dev_->importBuffers(count); }