From patchwork Wed Nov 10 10:08:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14515 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0C96BDB1C for ; Wed, 10 Nov 2021 10:08:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D708360369; Wed, 10 Nov 2021 11:08:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="U520UgS4"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D1D260128 for ; Wed, 10 Nov 2021 11:08:07 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 67-20020a1c1946000000b0030d4c90fa87so1533790wmz.2 for ; Wed, 10 Nov 2021 02:08:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OKJ83LIZfKI/rMNx9S3vEJe8UFBnXztTDF+aa2w7T3w=; b=U520UgS41XZXV+D75XUCAcdkx8QeCGPmZOm/LZlJkIvgmffhpRTT+Ao3AGV0aT99fn GXR9yuWR7ljIfH29j2YheFGpwuW4K0g76uq9IgzGePtSbH8h1gk08RgxGzDAods6AuGH o3f1CEPWUcOFJaZNDk2mOWmCkqq0PSQZPxTjzyqtHzdtRkbiAagv7/IcbG+MNIGv3YNd PXA7yp99TW1txkoNIVlkzxWP1juaqLxCdntkY1C0fJk+RZW6CVoB2sCXfW97qmDCT9sh hA3tgK+rd9YBkekydId96+AHYNR2mmx2yVihP3FJiGPS1E0GsFTHederneQrbDx78/3j BKyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OKJ83LIZfKI/rMNx9S3vEJe8UFBnXztTDF+aa2w7T3w=; b=rbcfoc06g1mXtrJ393KONALpOD+/8ZFQVcfQ++u3F1ViQ9KN0ZND4vn6p+jZTycOaz pfTDpbyoBBlLNJmv61h5d9LzpEYwQopK+ZkcH47K2AzxUqTf6Ar2HWkQYQ6pR9uZm3n9 Mes4MuDwMA1l++2hRNTikEDBY8Feafjlxh7Y/xiUAe8A4LWJeNhTqt/kMj7HUubguhDR PI9N7RyspKp44TpMXkTdu8C+GKLONVyRqkZD237wPhh82d7cuyIWyVuX62fCJgswsXIh mhrf9Gi7ZYoKGShPZ9vgz/KCvqt/XKs3s/qSrjQudtVBDOGNvs22HgeIp7DaSwCQk/5u MDpA== X-Gm-Message-State: AOAM532myyx5qwQeU9Dso5LCLYCJZ6kTzjc/J7DPMvHJCVQ2+x8nRSXc iGP2F42RBVXI6XMN0VZcUScQATHTHIV2w4A4 X-Google-Smtp-Source: ABdhPJyVzNgFeaphjIxrSg8vku8YiuG3xnMpYZJ55l8TAbiazieKsR3SvMphIiOsanYIhUW49LH6Pg== X-Received: by 2002:a05:600c:253:: with SMTP id 19mr14957968wmj.179.1636538887058; Wed, 10 Nov 2021 02:08:07 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:8747:6ef9:ee17:3d59]) by smtp.gmail.com with ESMTPSA id d11sm21969133wrs.38.2021.11.10.02.08.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 02:08:06 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 10:08:00 +0000 Message-Id: <20211110100802.349623-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/3] pipeline: raspberrypi: Add const qualifer in isRaw() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This function does not modify the pixFmt parameter, so use a const qualifier. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f227398a9..11d3c2b120dd 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,7 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, return deviceFormat; } -bool isRaw(PixelFormat &pixFmt) +bool isRaw(const PixelFormat &pixFmt) { /* * The isRaw test might be redundant right now the pipeline handler only From patchwork Wed Nov 10 10:08:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14516 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 532A8BDB1C for ; Wed, 10 Nov 2021 10:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 707D160362; Wed, 10 Nov 2021 11:08:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="G7ZJ8DU0"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0604B6034E for ; Wed, 10 Nov 2021 11:08:08 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id b184-20020a1c1bc1000000b0033140bf8dd5so1508956wmb.5 for ; Wed, 10 Nov 2021 02:08:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QTcjocxvUEcxxWS64VuvQEsn0E3guGnqlF2N2jhQXIA=; b=G7ZJ8DU0FOU5kEsKIMfDqw+ueTGPv0G457N9OFJPoi+rtT8Neg1/euRG/Sa5Rl1IDb VQW6ulVh15x8EWAfczygmIiiCbVl0V6/oQzSzOi8VqiFeZehThombHiOQ7Lyo5XK73pX yjRQPE7iexIWJqFrllzl7n9UcCRlw1dBSkcdC95dTB92PH5WEFQAzLJCw1ky4em8NeoJ pgM3H5JCYAetcyOPER4/tU+M6u+ocUaBOlhnE+CR2heCtLYUi8dUvzY/WDaNY/TNvhnV F81LrQ6quDp9TAtYWIVu8kBnTt8+mWK+so3b8N6DWTu9Ssrur/wDbzOWMFTWmUZmN4a+ F8Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QTcjocxvUEcxxWS64VuvQEsn0E3guGnqlF2N2jhQXIA=; b=00Nt+fDCbRntMRibbILQ426tQEbtMt8Mh5i0wQAzudgjzv5jZ9XWCYeqNHsNXjZ6Xt Hae5E4JQykRQ4EKaM5JuiPBSuTFRc2ottKDtqdAV/JbqQXePmlke1wavkiEfmO4hx5UD AhC6OATmmBlx+OwgiJtzohSYCp6c5/+L12UUqxfv4jLXbxpnZ6ZS2yewblM8yMllpa9X 0wq0/uRDzYJcvG+ZPACd3H3KCJWAQVYATqPZzoypO9eotm3Gq2y9wluIt3mkImcUHelD dbRye8r2kkKbI3FzSdQ4aIk6IlTpzkIImI45bbUXejSZVCH1OLQd5JixABWI5fXlZgg+ Fn+Q== X-Gm-Message-State: AOAM532/eKavn0iXAKUMC7qeztsC7+lHOxQihzPwwv/wkAzdKe5unolW BygiKhSM8ZwyD9lad17ssTDp2GfjfsvRsH0I X-Google-Smtp-Source: ABdhPJxrrooSvciRVg7K3SAmvB8fttkHSkt3WyNcbUqpdymDo+Oez/mRKN1gcr/enypf7LZSsdkvoA== X-Received: by 2002:a7b:c8c8:: with SMTP id f8mr14886473wml.49.1636538887596; Wed, 10 Nov 2021 02:08:07 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:8747:6ef9:ee17:3d59]) by smtp.gmail.com with ESMTPSA id d11sm21969133wrs.38.2021.11.10.02.08.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 02:08:07 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 10:08:01 +0000 Message-Id: <20211110100802.349623-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110100802.349623-1-naush@raspberrypi.com> References: <20211110100802.349623-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] pipeline: raspberrypi: Rework the internal buffer allocation scheme X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For simplicity, the pipeline handler would look at the maximum number of buffers set in the StreamConfiguration and allocate the same number of internal buffers for all device nodes. This would likely overallocate buffers for some nodes. Rework this logic to try and minimise overallcations without compromising performance. For ISP nodes, we only ever need 1 set of internal buffers, as the hardware runs synchronous with the requests and IPA. For Unicam nodes, allocate a minimum for 4 buffers (exported + internal), but also require at least 1 internal buffer. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- .../pipeline/raspberrypi/raspberrypi.cpp | 44 ++++++++++++++----- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 11d3c2b120dd..5f0f00aacd59 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1211,21 +1211,43 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); + unsigned int numBuffers; + bool rawStream = false; int ret; - /* - * Decide how many internal buffers to allocate. For now, simply look - * at how many external buffers will be provided. We'll need to improve - * this logic. However, we really must have all streams allocate the same - * number of buffers to simplify error handling in queueRequestDevice(). - */ - unsigned int maxBuffers = 0; - for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) - maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); + for (Stream *s : camera->streams()) { + if (isRaw(s->configuration().pixelFormat)) { + numBuffers = s->configuration().bufferCount; + rawStream = true; + break; + } + } + /* Decide how many internal buffers to allocate. */ for (auto const stream : data->streams_) { - ret = stream->prepareBuffers(maxBuffers); + if (stream == &data->unicam_[Unicam::Image] || + stream == &data->unicam_[Unicam::Embedded]) { + /* + * For Unicam, allocate a minimum of 4 buffers as we want + * to avoid any frame drops. If an application has configured + * a RAW stream, allocate additional buffers to make up the + * minimum, but ensure we have at least 1 set of internal + * buffers to use to minimise frame drops. + */ + constexpr unsigned int minBuffers = 4; + numBuffers = rawStream ? std::max(1, minBuffers - numBuffers) + : minBuffers; + } else { + /* + * Since the ISP runs synchronous with the IPA and requests, + * we only ever need one set of internal buffers. Any buffers + * the application wants to hold onto will already be exported + * through PipelineHandlerRPi::exportFrameBuffers(). + */ + numBuffers = 1; + } + + ret = stream->prepareBuffers(numBuffers); if (ret < 0) return ret; } From patchwork Wed Nov 10 10:08:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14517 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 845D5BDB1C for ; Wed, 10 Nov 2021 10:08:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C75386034A; Wed, 10 Nov 2021 11:08:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lXkI4E6F"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A07CE6034E for ; Wed, 10 Nov 2021 11:08:08 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id r8so3000096wra.7 for ; Wed, 10 Nov 2021 02:08:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l65rxfveEruSiHLvSJ6ZxHP4Un5MGATlNW0H1XExuqw=; b=lXkI4E6FhgHVJV1IrOX5sKYotSWsb2791INayvCwR402GDDPih+Yx/xQvzsRNgCS4m qMmBeYtTWUba6iqSEuwlgwWJQZ73MgyTKbhiJTaVKV6OZUuK2xwJxklCbukI4qK5AYJe 1uerXDG3J/1cRCNnDTlt8s9++/dYxWqBXr8zDnGc8D6yooigAmwgR2chkPhxjcj2U5ik Q8i+eeCld8kl68TryaYTZeocap7jqmTdfc0fxI+mutu1mVjOYY4FcMyJuPe5DueJXPxJ uo1Rd+wWMZaz5QrtNaN+H6Yy08r2b2QSHa02wjWjMY9cLRnT28bSL2cPAFQX7DgRvhxN 4r6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l65rxfveEruSiHLvSJ6ZxHP4Un5MGATlNW0H1XExuqw=; b=lOWT8/96zdnn8h8g1MHTgjY3XvlMOV/4u4nroPOjHx6t4p0eO/vTn9mU8tqmvaeKeT 1UtDqdHxTSMFfU/eG3gst8rMCy35YDG+k9Zfcmp4F7BImno7lm1UKdKLbyrReOP+0G9b +b9S+wysZbK5n9XVeJnOjsSKYVdRlg/zE2gLGJGkWphcuDKIvaZPzk8wo8L3ILgHRp52 J8z+A1Fkq+ej3v0WvsGb0d9RQN5dabq/icLH0U0Brpog5Se36PPbb1Vu2+zeNJxvoC/M kNNmDCQQtQCq+rEXA5ga6XLSVfEPTrE78NRf8HMwvSaCFBSWLvSkk47fXGZEXT1WFSd7 54oQ== X-Gm-Message-State: AOAM530p73iATXt+LkkgKlgAudZDKbACTDBFsSfsBOTTjRICk7b+36k7 JOd2YcMl8rTolnVYFGnuWxHbtIEc2ptO5Wwc X-Google-Smtp-Source: ABdhPJxDz4Fr4dIWmhI217CGdAd19BXUcHL+SL62t/QBGQylimx2s427U8EKopkMDHgoUBrYqd1V/w== X-Received: by 2002:adf:c70b:: with SMTP id k11mr17493046wrg.154.1636538888134; Wed, 10 Nov 2021 02:08:08 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:8747:6ef9:ee17:3d59]) by smtp.gmail.com with ESMTPSA id d11sm21969133wrs.38.2021.11.10.02.08.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 02:08:07 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 10:08:02 +0000 Message-Id: <20211110100802.349623-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110100802.349623-1-naush@raspberrypi.com> References: <20211110100802.349623-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/3] pipeline: raspberrypi: Increase the V4L2BufferCache slot allocations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roman Stratiienko Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If a stream is marked as external, double the number of V4L2BufferCache slots that are allocated. This is to account for additional buffers that may be allocated directly by the application. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/libcamera/pipeline/raspberrypi/rpi_stream.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b3265d0e8aab..67901936d6b6 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -107,6 +107,14 @@ int Stream::prepareBuffers(unsigned int count) count = bufferMap_.size(); } + /* + * If this is an external stream, we must allocate slots for buffers that + * might be externally allocated. We have no indication of how many buffers + * may be used, so this might overallocate slots in the buffer cache. + */ + if (isExternal()) + count = count * 2; + return dev_->importBuffers(count); }