From patchwork Wed Jun 29 10:30:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16427 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77D16BE173 for ; Wed, 29 Jun 2022 10:30:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2823A6563B; Wed, 29 Jun 2022 12:30:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498628; bh=IIm2cZngUsiUEYkBau8iCGgMAohwfeH0xvQ4TnD+wKg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=uaB1mU5kbZAkVKvQg8+bqTvfdblaRXJ8hPwwpQ5QecCqru4sc+Rr9COe0CyWyOhuB s/0SUGTfX+ib+n1UYoPqXZ3XadmfKKIV5aZ3BcV0hjIq7ICJNVG8NVvbpjmTPhJmcy T9CivgJ/xR9YwfA6bkCkktlEvo/oBhtmtM8nmZPBygDBw/Qg6d7JCTrNGOZFHSs+Cc nh30pe67iPGHmfi5LGhcZuV+Ml7K/uQ5fKu620Uelwcn+k/e5iDfB78Ty7MYO9x3SH k/22G1LtM/4dLRpPlCPzXwQFBXl+cnvzyTxWYXrl/xthP9/Gw6d0a3OVBXX2K5N3wq 2HmopdWccSgYw== Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2038060412 for ; Wed, 29 Jun 2022 12:30:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="LA744NiW"; dkim-atps=neutral Received: by mail-pg1-x52b.google.com with SMTP id h192so14932003pgc.4 for ; Wed, 29 Jun 2022 03:30:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9ri2f8MX8DhxAdhDF8VN80ETHfxE4jKgxPQyULCGKnc=; b=LA744NiW7UlzW9vIfZDj7/v552FxavCA9vttsDhT8tKUQVXxz+CaGNFow3ZFCXxIXl 6hquxxvbIs3DqM7UcviRd8WMgfQpfLTo28Aq5zSD9VnhtBz9ywn09mowu/IH9J/1YPlu BNXWnotvwLCOuoUvjh98Rnv9bEh3BpALu8w+U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9ri2f8MX8DhxAdhDF8VN80ETHfxE4jKgxPQyULCGKnc=; b=ATqKhfEm9h5ZgA0MtyJ2KAqkA651W/fBVirtKR6Ixd+nxt0XibjBpiJY+bge84b3LL BkINv4BtkDH28ST3gYNraInhz+3rms7BkYH0BeuzkjZy92RP+ku/znTeWrCquTeJokmN 1sKpceBbRtAVaSvYfnJJdvfAMNqJtATA+LBcJ2lh1q2Qru2xGMllV5YestwnCp33aHk6 uFCCP1xm7C1LCT71XYCyyMApWFnSRa3/97Jw6jM6akwm6/t3NTosz1UxTnCa6d9u+AMF Gn31n6Rf8tvOzhZhmKdGbCGo+sC5i19ZbfY2OCM9LzSGczUapFfmhaUagKvCv18PDhGh VLNA== X-Gm-Message-State: AJIora+sDBwgO0QvK0ocx18M/y5BmKlFZ1wXjTOtc+7rohMJ1G2qDnrq 4VCamAYhvvef5VdjuonkU6N/1NnbhzUaXLzK X-Google-Smtp-Source: AGRyM1uy9DIBwM1WeWKSeA3rZw+yn/gmN24EH+cz1N9pmtlL4HVorwyHOx1iH5B5Jpy4T1JFjaH0ow== X-Received: by 2002:a63:ff19:0:b0:403:7c60:ae96 with SMTP id k25-20020a63ff19000000b004037c60ae96mr2378351pgi.466.1656498623544; Wed, 29 Jun 2022 03:30:23 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:23 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:10 +0000 Message-Id: <20220629103018.4025635-2-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/9] Add StreamRole in StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang To let ipu3 pipeline handler understand the usage of streams, this patch adds StreamRole member variable in StreamConfiguration, which should be set by the (Android) adapter. Signed-off-by: Harvey Yang --- include/libcamera/stream.h | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index f0ae7e62..c9e79226 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -22,6 +22,15 @@ namespace libcamera { class Camera; class Stream; +enum StreamRole { + Raw, + StillCapture, + VideoRecording, + Viewfinder, +}; + +using StreamRoles = std::vector; + class StreamFormats { public: @@ -50,6 +59,8 @@ struct StreamConfiguration { std::optional colorSpace; + StreamRole streamRole; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } @@ -61,15 +72,6 @@ private: StreamFormats formats_; }; -enum StreamRole { - Raw, - StillCapture, - VideoRecording, - Viewfinder, -}; - -using StreamRoles = std::vector; - class Stream { public: From patchwork Wed Jun 29 10:30:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16428 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4BFDFBE173 for ; Wed, 29 Jun 2022 10:30:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2972B6563F; Wed, 29 Jun 2022 12:30:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498629; bh=cnT2I2kXHoViJJi+mchR8wLp6h1hHGjuDpJyjvEiPGw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wtWRWRcfSR16jTrXnfutsmtgeUDle4UDxmlcF4QI1Hyg3NuERPqRlLmzFmBgPNDYT cJKbnFCxWy3KOnrJ6Ylm0zgokA3qb+8IjYj4o9cpDSgWThyKEgsRIze4gFJD5oCH8k H9HkNYk8RreRrfQXcSl2BLx9DDrKD5vHneE477PMDHmJIYFkMr8I7ktimVOmnz7wmY EPOvlSqz0NX/wi4gdORfgmVD/TqGOOkm2f5+ucTFLdxiF4GW3e7fWm/tWrGsrQlaM0 RCXALvjZ4ObXxRiU3iC8ffdZqdE/nhOPpNf280mr620OafoCPlFtzb6nPYKi0WL7GJ dmz+wuCjLTqpg== Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A46DC60412 for ; Wed, 29 Jun 2022 12:30:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="gdxmm1yZ"; dkim-atps=neutral Received: by mail-pl1-x62b.google.com with SMTP id k14so13707881plh.4 for ; Wed, 29 Jun 2022 03:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SIwkgrFVZoMpv3fQMHYD/oCCOcCeDkunHhcLDNTInRI=; b=gdxmm1yZd8xdinJxTIzL6gpVEwejYIkBJE3TxScuriWB6H3MQtmtwy5vzfgmZW8UJo U7hGFoh72hk6jfIVE14FEtGGHqtk5e0uNir46nYZ/tLcF5FIzYkdmmj46gt4wJoo/Bd1 Hptice/40ig9C14a/LoG8FN0VADhAGpVNm5cw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SIwkgrFVZoMpv3fQMHYD/oCCOcCeDkunHhcLDNTInRI=; b=qY6iodzFdNwFH/rEu5F6ueePr2Wu2eKh8hirkr1aLf0PIBRtHoeTbAYz+dhO8LNgzG ZFV8/0DSS/SSe9ngO+x7xNu1B2RxsdIMfKeFI7ketgh/Mz9pB371vne8AthA7nxVrPgR EQjX0tw+f3dVZVIyWUR9VY9GOzJjfYodX6pDJtzsxXZYtyqQ5RjXYSuksWOglYRg8TVk 19JhP3KPWn2xgT6SbhfPk+PTi2p7cylCAcd2dSMw0+bDO1kSG/bhnJNdn7S9f4XcvJzy USLyUlClBdUDGP1SdZ2eyzIx9wU7IXSF0+1Z25Rn86O7hW0yQ0etlbt3ek1AvemyGtN7 o3lA== X-Gm-Message-State: AJIora+8aJiBtCo/cON24qB7sjC1VGZRfVhIICGarpFKeonmGl1deEtl FOwx+10kEOWT1GQw1ooczKwQfZgjVJ9UCJSX X-Google-Smtp-Source: AGRyM1u7uyuYXadKQiykiZbedaA4/5Q8iLXlgcdAJ0o22OfSPY3yXtSYQCScJV4tp6ZXOum1Tqt0nQ== X-Received: by 2002:a17:902:bb8d:b0:168:e48d:86bc with SMTP id m13-20020a170902bb8d00b00168e48d86bcmr9881545pls.93.1656498624645; Wed, 29 Jun 2022 03:30:24 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:24 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:11 +0000 Message-Id: <20220629103018.4025635-3-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/9] ipu3: Allow only one camera being started X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang As we hardly have use cases/applications that need both cameras at the same time, this patch adds a rule that only one camera can be started one time. This also allows the following patches that use both imgus to process frames from one single camera. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index fd989e61..c943ee6a 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -166,6 +166,8 @@ private: MediaDevice *cio2MediaDev_; MediaDevice *imguMediaDev_; + Camera *inUseCamera_ = nullptr; + std::vector ipaBuffers_; }; @@ -765,6 +767,14 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { + /* + * Enforce that only a single camera can be used at a time to use both + * ImgUs on the camera, so that StillCapture stream can adopt another + * set of configuration. + */ + if (inUseCamera_ && inUseCamera_ != camera) + return -EBUSY; + IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; @@ -781,6 +791,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) return ret; + inUseCamera_ = camera; + ret = data->ipa_->start(); if (ret) goto error; @@ -808,6 +820,8 @@ error: freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); + inUseCamera_ = nullptr; + return ret; } @@ -826,6 +840,8 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + + inUseCamera_ = nullptr; } void IPU3CameraData::cancelPendingRequests() From patchwork Wed Jun 29 10:30:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16429 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A637EBE173 for ; Wed, 29 Jun 2022 10:30:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6835065642; Wed, 29 Jun 2022 12:30:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498631; bh=oIrRxgWxnXvAKLo4F65Dfr5GH6EKBHOAv4e5Gty+XTI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ZoN/ONqsl8QHCFR9mQeGvU7PvQBSCA/KMu/v7mgOP2EgjDYg/vfdHS+x+K67RgGZM 1zn33L2ySEe/PYgSr/Opp8HKFjLpZdpvmAKkX9Q0PcoFsDkX4PXPjrCyTQD6vWOz3Q Oa3B9U0ivbdmipQr6eixG8I4HL842GXzERjFFHUw+++wkCkqPxyPDYV/xaly4TU4Fe kqSIU4xdf/g8qw6Uhi5SrvoqsbarjhODRRcc9CaeFgF9+gT4lVhkmDNa2UfYf2XOBI qHOfezI1KZKYx11hZdmFC9YY/S5wlsDLFnl+CBue8JoU9lj6+OW3YqX5vqyUt7870t Fi4bLmWCEXa9w== Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F26B565631 for ; Wed, 29 Jun 2022 12:30:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VA41+dtq"; dkim-atps=neutral Received: by mail-pg1-x532.google.com with SMTP id z14so14950450pgh.0 for ; Wed, 29 Jun 2022 03:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pj21KPq7U93iREIWfwjAbXuLCLUn5kburhNyE5CWBnM=; b=VA41+dtqT6hNaSxFhdq54jc9V1mTrZ1pEc3thV3wjBwdUoFYBpB0cwlROLsDKlG3Qd GeGSpY3fEwMEAtus6DWwI9J9cuuoOvnVD7xqbA7iBVu2xLf1n8NlgUksDOo35rBac+Af UN07q/6UgoHvOQqUTe1ZvD2FLFZICVHBxPuPc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pj21KPq7U93iREIWfwjAbXuLCLUn5kburhNyE5CWBnM=; b=K8Jz+UuFVBp4lKmis1fECPNvVAX858ckachkh7eEebcOZVtXgJfLfNRguLJ/ffRZFF R+6wuDLagGjR4jpbl9ja1GxGwoO+SUroMBqWk9RDvwASMaEDI0NzReQfSGuBHICOgitc ERdIa7kbbtf3huvtdEtBjc5jfAfg5u7lxhLxRmBMuPqQ28hAjJO7ScJfCzvKkade72Mh 5XP96CR0fl+UIWUIOfHNYjHdmaPFpfET4SQsfzsYHMoVcy+bhoD9CXTlXOMejUXFRWNA MhPjm2Wpr4hieUyDBDg1w8o2UZ6vBp/EUoO9xJmJakOK4tXlTqnw1MV4o3D7oit7MqL5 /Z+A== X-Gm-Message-State: AJIora83PGJCqG3qp3Zfw9KrIpx7DuKS8Pds1hyAGgAEc2vCO4SyLDYU K8Bp+rQec9LrrnFdfd9tQ7pa8ud4AHS/Y9m2 X-Google-Smtp-Source: AGRyM1ul/Sw8oZjQCKWpKMH2YS0AaEu1LOOROO6KWpaDMh5mora38++Y77bxALD8/k/Tn6KgQaJdSA== X-Received: by 2002:a63:751b:0:b0:40c:9c39:c890 with SMTP id q27-20020a63751b000000b0040c9c39c890mr2391898pgc.302.1656498626298; Wed, 29 Jun 2022 03:30:26 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:25 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:12 +0000 Message-Id: <20220629103018.4025635-4-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/9] ipu3: Use imgu0 as default X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang With only one camera being started, we can always use imgu0 to process frames (for video/preview). In the following patches, we'll use imgu1 for still capture if needed. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 86 ++++++++++++++++------------ 1 file changed, 48 insertions(+), 38 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index c943ee6a..e219f704 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -64,7 +64,8 @@ public: void frameStart(uint32_t sequence); CIO2Device cio2_; - ImgUDevice *imgu_; + ImgUDevice *imgu0_; + ImgUDevice *imgu1_; Stream outStream_; Stream vfStream_; @@ -406,7 +407,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu_->calculatePipeConfig(&pipe); + pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; @@ -518,7 +519,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; V4L2DeviceFormat outputFormat; int ret; @@ -560,7 +560,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream which is for raw capture, in which case no buffers will * ever be queued to the ImgU. */ - ret = data->imgu_->enableLinks(true); + ret = imgu0_.enableLinks(true); if (ret) return ret; @@ -610,7 +610,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (imguConfig.isNull()) return 0; - ret = imgu->configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig, &cio2Format); if (ret) return ret; @@ -624,12 +624,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (stream == outStream) { mainCfg = &cfg; - ret = imgu->configureOutput(cfg, &outputFormat); + ret = imgu0_.configureOutput(cfg, &outputFormat); if (ret) return ret; } else if (stream == vfStream) { vfCfg = &cfg; - ret = imgu->configureViewfinder(cfg, &outputFormat); + ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; } @@ -641,13 +641,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * be at least one active stream in the configuration request). */ if (!vfCfg) { - ret = imgu->configureViewfinder(*mainCfg, &outputFormat); + ret = imgu0_.configureViewfinder(*mainCfg, &outputFormat); if (ret) return ret; } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu->imgu_->controls()); + ControlList ctrls(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. @@ -657,7 +657,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) */ ctrls.set(V4L2_CID_IPU3_PIPE_MODE, static_cast(IPU3PipeModeVideo)); - ret = imgu->imgu_->setControls(&ctrls); + ret = imgu0_.imgu_->setControls(&ctrls); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; @@ -691,9 +691,9 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, unsigned int count = stream->configuration().bufferCount; if (stream == &data->outStream_) - return data->imgu_->output_->exportBuffers(count, buffers); + return imgu0_.output_->exportBuffers(count, buffers); else if (stream == &data->vfStream_) - return data->imgu_->viewfinder_->exportBuffers(count, buffers); + return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); @@ -711,7 +711,6 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, int PipelineHandlerIPU3::allocateBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - ImgUDevice *imgu = data->imgu_; unsigned int bufferCount; int ret; @@ -721,26 +720,26 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->rawStream_.configuration().bufferCount, }); - ret = imgu->allocateBuffers(bufferCount); + ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; - for (const std::unique_ptr &buffer : imgu->paramBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.paramBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } - for (const std::unique_ptr &buffer : imgu->statBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.statBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -760,7 +759,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) data->ipa_->unmapBuffers(ids); ipaBuffers_.clear(); - data->imgu_->freeBuffers(); + imgu0_.freeBuffers(); return 0; } @@ -777,9 +776,18 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; int ret; + imgu0_.input_->bufferReady.connect(&data->cio2_, + &CIO2Device::tryReturnBuffer); + imgu0_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.viewfinder_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.param_->bufferReady.connect(data, + &IPU3CameraData::paramBufferReady); + imgu0_.stat_->bufferReady.connect(data, + &IPU3CameraData::statBufferReady); /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -807,19 +815,24 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; - ret = imgu->start(); + ret = imgu0_.start(); if (ret) goto error; return 0; error: - imgu->stop(); + imgu0_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); + imgu0_.input_->bufferReady.disconnect(); + imgu0_.output_->bufferReady.disconnect(); + imgu0_.viewfinder_->bufferReady.disconnect(); + imgu0_.param_->bufferReady.disconnect(); + imgu0_.stat_->bufferReady.disconnect(); inUseCamera_ = nullptr; return ret; @@ -834,13 +847,19 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); - ret |= data->imgu_->stop(); + ret |= imgu0_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + data->imgu0_->input_->bufferReady.disconnect(); + data->imgu0_->output_->bufferReady.disconnect(); + data->imgu0_->viewfinder_->bufferReady.disconnect(); + data->imgu0_->param_->bufferReady.disconnect(); + data->imgu0_->stat_->bufferReady.disconnect(); + inUseCamera_ = nullptr; } @@ -1184,7 +1203,8 @@ int PipelineHandlerIPU3::registerCameras() * only, and assign imgu0 to the first one and imgu1 to the * second. */ - data->imgu_ = numCameras ? &imgu1_ : &imgu0_; + data->imgu0_ = &imgu0_; + data->imgu1_ = &imgu1_; /* * Connect video devices' 'bufferReady' signals to their @@ -1198,16 +1218,6 @@ int PipelineHandlerIPU3::registerCameras() &IPU3CameraData::cio2BufferReady); data->cio2_.bufferAvailable.connect( data.get(), &IPU3CameraData::queuePendingRequests); - data->imgu_->input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); - data->imgu_->output_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->viewfinder_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->param_->bufferReady.connect(data.get(), - &IPU3CameraData::paramBufferReady); - data->imgu_->stat_->bufferReady.connect(data.get(), - &IPU3CameraData::statBufferReady); /* Create and register the Camera instance. */ const std::string &cameraId = cio2->sensor()->id(); @@ -1300,14 +1310,14 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) FrameBuffer *outbuffer = it.second; if (stream == &outStream_) - imgu_->output_->queueBuffer(outbuffer); + imgu0_->output_->queueBuffer(outbuffer); else if (stream == &vfStream_) - imgu_->viewfinder_->queueBuffer(outbuffer); + imgu0_->viewfinder_->queueBuffer(outbuffer); } - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Wed Jun 29 10:30:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16430 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CCE88BE173 for ; Wed, 29 Jun 2022 10:30:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 575E765648; Wed, 29 Jun 2022 12:30:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498634; bh=frMGXmQgQkeQXSJMRCgTkMQ6+oAY30aS/uEiFMkQG+0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=h2GgL24ywoxSRsS4lxCCF5LkotAAFLqosKpufsai7t/BgfnNuHZeX9VI1FE2ffnhQ Y/dE9xfhbt3HLddfL5/Y1PKDyxB2BWWs2mrCzH33CSttttgWxODIwcxh+e05GkaLyW y+gCAXv6bR1w/uOSWdJcMbCCugoVuMMbvbEuWGyMLNgaB0c4zdi2sPdWxS1gj2DHey w0hMWXZBbXk00Ky7CrqFmid9CeW5JFDdY1GfRiHED1ik+vbvFQoEh044a3KF8DG/nE L+0ZApd8owIzuwIx5Xab2DKuVHkrqmAemRozBLzSfaiqVdJioRAGbyju4b/Pt6WAEv q6+o0I6Jg7Lyw== Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 138726563E for ; Wed, 29 Jun 2022 12:30:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="lxokORti"; dkim-atps=neutral Received: by mail-pf1-x436.google.com with SMTP id d17so14666718pfq.9 for ; Wed, 29 Jun 2022 03:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Bm9QfL5yVumPD3LxUYZj9M11EUpxJ2l+LkPlvWOTvvI=; b=lxokORtiTbEBE9Kgl7BT/fm7mD+LAcsGJGlaNh49QrOQ/yYe7lBSp+cMCRmBSaMRmW 9xN7hSi5scLO+uiVmPHzM21t2ygf1I2HXwnWsSR0SVcJ7/K9f6+QQglRFcxkKPaKgydN vBs8UzszujAQTr+TfxjrKK6NO56pYjBr1Cu6o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Bm9QfL5yVumPD3LxUYZj9M11EUpxJ2l+LkPlvWOTvvI=; b=2nWXY6bjGamiX+MuDsceygQCDhrmn8wsvVNtp/D+lIKmUrgaihRKgM4lYlGECGvHIK JnSrPKYOSFUDwFNnsESFW1T0NmA6A2mB8keaitJ1ErePkAVFe49l5t37bxcD3ULliEgx x38jJp9oSy4IGjQdPbnseSFKyzMJL0i+f/OFpOEc/hLas+vwOfOxlzkp+u2muLYflgqo FYOhKf+0BTbsqrvhCpYHDYMs/YrFLgOKxwZZpNr1EwIP5zpGDKtWGjGycjcePnz55aBh AD4ivOcVV3OYzfpbzT/T12qerB4c71DfZn7EH/zR32FTiqsb2x7W7L/QerKAmw17MWfO ZNfQ== X-Gm-Message-State: AJIora9D3xXnZHMfK0Y7BoMqNzhqHLTroreh+DXQY0Npi6Wwl+VcsIEL mrjxtmnbsl0VGAL2rryuR2oI8EuMqqvfjM19 X-Google-Smtp-Source: AGRyM1vkkN/TnEluT0pya00m74p7lQANCS3jYhhAcLvsegibe5B6LZQ/nqB9ORLWYugxtLB+ItX0XQ== X-Received: by 2002:a63:8141:0:b0:40d:28fc:440f with SMTP id t62-20020a638141000000b0040d28fc440fmr2466104pgd.12.1656498627408; Wed, 29 Jun 2022 03:30:27 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:27 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:13 +0000 Message-Id: <20220629103018.4025635-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/9] ipu3: Add StillCapture stream and imgu1 param buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch adds the StillCapture stream and imgu1 param buffers. The following patches will enable imgu1 to handle StillCapture stream specifically, when the imgu0 is needed to handle video/preview streams. Signed-off-by: Harvey Yang Reviewed-by: Umang Jain --- src/libcamera/pipeline/ipu3/frames.cpp | 23 +++++++++++++++++++++-- src/libcamera/pipeline/ipu3/frames.h | 8 +++++++- src/libcamera/pipeline/ipu3/ipu3.cpp | 20 +++++++++++++++++++- 3 files changed, 47 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/frames.cpp b/src/libcamera/pipeline/ipu3/frames.cpp index a4c3477c..f9b705a5 100644 --- a/src/libcamera/pipeline/ipu3/frames.cpp +++ b/src/libcamera/pipeline/ipu3/frames.cpp @@ -23,7 +23,8 @@ IPU3Frames::IPU3Frames() } void IPU3Frames::init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers) + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers) { for (const std::unique_ptr &buffer : paramBuffers) availableParamBuffers_.push(buffer.get()); @@ -31,6 +32,9 @@ void IPU3Frames::init(const std::vector> ¶mBuff for (const std::unique_ptr &buffer : statBuffers) availableStatBuffers_.push(buffer.get()); + for (const std::unique_ptr &buffer : captureParamBuffers) + availableCaptureParamBuffers_.push(buffer.get()); + frameInfo_.clear(); } @@ -38,6 +42,7 @@ void IPU3Frames::clear() { availableParamBuffers_ = {}; availableStatBuffers_ = {}; + availableCaptureParamBuffers_ = {}; } IPU3Frames::Info *IPU3Frames::create(Request *request) @@ -54,14 +59,22 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) return nullptr; } + if (availableCaptureParamBuffers_.empty()) { + LOG(IPU3, Debug) << "Capture parameters buffer underrun"; + return nullptr; + } + FrameBuffer *paramBuffer = availableParamBuffers_.front(); FrameBuffer *statBuffer = availableStatBuffers_.front(); + FrameBuffer *captureParamBuffer = availableCaptureParamBuffers_.front(); paramBuffer->_d()->setRequest(request); statBuffer->_d()->setRequest(request); + captureParamBuffer->_d()->setRequest(request); availableParamBuffers_.pop(); availableStatBuffers_.pop(); + availableCaptureParamBuffers_.pop(); /* \todo Remove the dynamic allocation of Info */ std::unique_ptr info = std::make_unique(); @@ -71,7 +84,9 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) info->rawBuffer = nullptr; info->paramBuffer = paramBuffer; info->statBuffer = statBuffer; + info->captureParamBuffer = captureParamBuffer; info->paramDequeued = false; + info->captureParamDequeued = false; info->metadataProcessed = false; frameInfo_[id] = std::move(info); @@ -84,6 +99,7 @@ void IPU3Frames::remove(IPU3Frames::Info *info) /* Return params and stat buffer for reuse. */ availableParamBuffers_.push(info->paramBuffer); availableStatBuffers_.push(info->statBuffer); + availableCaptureParamBuffers_.push(info->captureParamBuffer); /* Delete the extended frame information. */ frameInfo_.erase(info->id); @@ -102,6 +118,9 @@ bool IPU3Frames::tryComplete(IPU3Frames::Info *info) if (!info->paramDequeued) return false; + if (!info->captureParamDequeued) + return false; + remove(info); bufferAvailable.emit(); @@ -131,7 +150,7 @@ IPU3Frames::Info *IPU3Frames::find(FrameBuffer *buffer) return info; if (info->rawBuffer == buffer || info->paramBuffer == buffer || - info->statBuffer == buffer) + info->statBuffer == buffer || info->captureParamBuffer == buffer) return info; } diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h index 6e3cb915..8fcb8a14 100644 --- a/src/libcamera/pipeline/ipu3/frames.h +++ b/src/libcamera/pipeline/ipu3/frames.h @@ -36,16 +36,20 @@ public: FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *captureParamBuffer; + ControlList effectiveSensorControls; bool paramDequeued; + bool captureParamDequeued; bool metadataProcessed; }; IPU3Frames(); void init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers); + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers); void clear(); Info *create(Request *request); @@ -61,6 +65,8 @@ private: std::queue availableParamBuffers_; std::queue availableStatBuffers_; + std::queue availableCaptureParamBuffers_; + std::map> frameInfo_; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e219f704..a201c5ca 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -70,6 +70,7 @@ public: Stream outStream_; Stream vfStream_; Stream rawStream_; + Stream outCaptureStream_; Rectangle cropRegion_; bool supportsFlips_; @@ -696,6 +697,8 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); + else if (stream == &data->outCaptureStream_) + return imgu1_.output_->exportBuffers(count, buffers); return -EINVAL; } @@ -718,11 +721,17 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->outStream_.configuration().bufferCount, data->vfStream_.configuration().bufferCount, data->rawStream_.configuration().bufferCount, + data->outCaptureStream_.configuration().bufferCount, }); ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; + ret = imgu1_.allocateBuffers(bufferCount); + if (ret < 0) { + imgu0_.freeBuffers(); + return ret; + } /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; @@ -737,9 +746,14 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } + for (const std::unique_ptr &buffer : imgu1_.paramBuffers_) { + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + } + data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_, imgu1_.paramBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -760,6 +774,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) ipaBuffers_.clear(); imgu0_.freeBuffers(); + imgu1_.freeBuffers(); return 0; } @@ -1143,6 +1158,7 @@ int PipelineHandlerIPU3::registerCameras() &data->outStream_, &data->vfStream_, &data->rawStream_, + &data->outCaptureStream_, }; CIO2Device *cio2 = &data->cio2_; @@ -1318,6 +1334,8 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) imgu0_->param_->queueBuffer(info->paramBuffer); imgu0_->stat_->queueBuffer(info->statBuffer); imgu0_->input_->queueBuffer(info->rawBuffer); + + info->captureParamDequeued = true; } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Wed Jun 29 10:30:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16431 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CA51BE173 for ; Wed, 29 Jun 2022 10:30:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C59D365635; Wed, 29 Jun 2022 12:30:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498636; bh=522BI9487O+gF2B9gn8dxLTi51VfnY5JKZuEg2Hrl6s=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ax3wUw8A7jb7kgPgX4ZAd/G9p76W3dnI3yDs6BSkCJASTKJ50sBS7RzYRlr+9tthw DzcSq5XE1pUEOJErP/SD+fLqYVTpkYsuuEXkJ0Til7B5FFX7XyE7If0Wv4vJXPeivD a1NDKeAp77Pq86IHpiQVKZoFFWbn0QLR1yfEcLCIU3KmkNHqWc/lTg+q7JnazoRrYx 83OtGyYPNaM6KX12of+1Q1y+O9HDqBcPn5fw189dcsuWwfBml571HBXbUtFOeezbIy i2FjBAUfRzt5bC3TTyTZibuIc9xlHkd1dZahmDtQxVetu4QhtadLDWB66b6d9RLQNu caFfC0tldMKqQ== Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D10E65631 for ; Wed, 29 Jun 2022 12:30:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ZkixFWKu"; dkim-atps=neutral Received: by mail-pj1-x1031.google.com with SMTP id h9-20020a17090a648900b001ecb8596e43so15576075pjj.5 for ; Wed, 29 Jun 2022 03:30:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UbbNcnO9GasdwFGniWaaPqU96QfmrN4OHi5sJ44cEWE=; b=ZkixFWKuTl31TE0ksaDMc1LJzK+RFqvuM+Unu7bHtR7psKqQ2AwNtb0cLndRbi0+uz 4ikyaw5IIQX/KCBMQq0TdX54n1qe5HiFfTaypAtF5rGOtOJHSzk8tvvrxMu/VAcdB4bF V1uOawpkESW2lV4MVQmEq5O4t3Ye8V+2OH9EA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UbbNcnO9GasdwFGniWaaPqU96QfmrN4OHi5sJ44cEWE=; b=QFm1x7aord0x37IqipJY2QrJrXdal8k6sd4IRafRIYqFYL+lOzzHLBeABG5FChQKpC oUaPJx7DaVd027OWNq2tH3XnqvaigFAuzDj8IAmDwm3FEJ2cZ/1iih+n/OY4vJyDoNkK IRJqyyYjM5tbxQjwdEH+wpW/Qi55v/MNwlt7f8mEfylp9c3+hfvKhtU2/SrL1QJ4XCM+ COVwILrQU2+w6HLwNhTcogz67BPwj0F66OA0CMM7S/FX8RLiW0HWUGsq5jG0z/Ydy2k+ xjrKZfUfseRgPcAX6q1lzpFEJFKGpQmGPxkT5kzmFoBfNDSSD1eqjMUneo6wCWro1ngh mlJA== X-Gm-Message-State: AJIora/2J3U4Ya6AGzZ7hOzU8hSI3wwjTokdkEzNx0zBrgwSwycwSRDS FPAL64jdlyNINOghJxFv3bAZ+MdcL/THw56e X-Google-Smtp-Source: AGRyM1tUH9bdW8jbk9nkNuSyAi4GITFqr9+u4HHNaEQ0WElQ6BAKQeIklzffvGmz0tPhNbX50xPKgg== X-Received: by 2002:a17:90b:4b41:b0:1ec:cb06:2fa3 with SMTP id mi1-20020a17090b4b4100b001eccb062fa3mr3045199pjb.55.1656498628641; Wed, 29 Jun 2022 03:30:28 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:28 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:14 +0000 Message-Id: <20220629103018.4025635-6-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/9] ipu3: Update IPAIPU3Interface::fillParamsBuffer with captureBufferId X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch updates the ipa interface |IPAIPU3Interface::fillParamsBuffer| with additional |captureBufferId| to fill the param buffer for the StillCapture stream as well. Signed-off-by: Harvey Yang --- include/libcamera/ipa/ipu3.mojom | 2 +- src/ipa/ipu3/ipu3.cpp | 21 ++++++++++++++++++--- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- 3 files changed, 21 insertions(+), 5 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d1b1c6b8..d94c344e 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -31,7 +31,7 @@ interface IPAIPU3Interface { unmapBuffers(array ids); [async] queueRequest(uint32 frame, libcamera.ControlList controls); - [async] fillParamsBuffer(uint32 frame, uint32 bufferId); + [async] fillParamsBuffer(uint32 frame, uint32 bufferId, uint32 captureBufferId); [async] processStatsBuffer(uint32 frame, int64 frameTimestamp, uint32 bufferId, libcamera.ControlList sensorControls); }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dd6cfd79..149a3958 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -146,7 +146,8 @@ public: void unmapBuffers(const std::vector &ids) override; void queueRequest(const uint32_t frame, const ControlList &controls) override; - void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) override; void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -508,12 +509,14 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) /** * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number - * \param[in] bufferId ID of the parameter buffer to fill + * \param[in] bufferId ID of the video parameter buffer to fill + * \param[in] captureBufferId ID of the capture parameter buffer to fill * * Algorithms are expected to fill the IPU3 parameter buffer for the next * frame given their most recent processing of the ImgU statistics. */ -void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) +void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -536,6 +539,18 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + for (auto const &algo : algorithms_) + algo->prepare(context_, params); + + // TODO: use different algorithms to set StillCapture params. + it = buffers_.find(captureBufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find capture param buffer!"; + return; + } + mem = it->second.planes()[0]; + params = reinterpret_cast(mem.data()); + params->use = {}; for (auto const &algo : algorithms_) algo->prepare(context_, params); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a201c5ca..a13fb881 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1424,7 +1424,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie(), + info->captureParamBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) From patchwork Wed Jun 29 10:30:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16432 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 71A40BE173 for ; Wed, 29 Jun 2022 10:30:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 243566564A; Wed, 29 Jun 2022 12:30:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498637; bh=tCsDkvLIRPMp8NkbSM+fZjmRxr1XZq/dIBbbebw9z2U=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=s2LshpeHrg52a2he6ER54HPkmPRfcu8Vp3Dhs6+6UAZ7mfOCexE6kNyOvvcdGct6D CfcFDzLje3CH+jn7N9pKTh1tUruVSSgkacp0SPzTHVNDCt0sxWwvtbfGEt82fkCkqV FYNY4T06d4P0kF73lB1bNub1x8E2j/Dl+vk2RvZc5fbSwB6K7TKAlSoW9GBcW97TPX +uMl0yl3hvAgkygQrFkrN7wX4quwlgKVapS8XfxpdvdtLga3x8EAttJYhbn+tr/LOb pL0JRtvrho/Hp/jb5UgJ0No7MbIYsgRpE7qvn7d5STtz4sIXVgXbql7pT63U6aOEbE Z//utCvxCcX8g== Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9694265643 for ; Wed, 29 Jun 2022 12:30:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="GWwBpi/Z"; dkim-atps=neutral Received: by mail-pg1-x52f.google.com with SMTP id e63so14913392pgc.5 for ; Wed, 29 Jun 2022 03:30:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lLE8xkogtcdUmvJ6QKLyJkUAIsGf0UlLlM995CLC/gY=; b=GWwBpi/ZunZdG5o0f8oPs6ozzxn3cyZdtAih6cuNpto0HOIxXil65TTZNVRiA5UQ9O OT670Udpb6MhDd1lLJ7+43jzb1/Ro7l4Ttd6x/R9lQ5fbwM5ydmk4/wDyqlR4Otxx2cE wyX1wmTO1e9N8FDARKr/L3dLTKov6XzIBbRiQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lLE8xkogtcdUmvJ6QKLyJkUAIsGf0UlLlM995CLC/gY=; b=4sj0hZVBFlzBvphCnpmvYgfMejzV15ckEAqZyV+HlOwtAIcBLHhBC5b3iXe234b+V9 W4gtII6aMuBcMMqan6WXVyIBLGuziYtwzdizO5bOB6hFwl3Rzipltrv9/WjgQ4zZKJKh r8DZtH3igVL2j+UC0c/WBNvm5VajYvmvRqWF8ZJ7oG/oJX+39H2Jv+gMiYtG+Rwd5Lyw +MNUexOfjO4wQNnICglYcVF/td8uE7cU49bPs3AB9yQvydQ5LcoQ/iCd8eEVDds848EF kuW8zC+5hteErNpa3Bac90XhwqEThS2DDMBqo3zjNxPRYi6ie4qB+uHmqdOqbRcTossb TYVQ== X-Gm-Message-State: AJIora9A1RmyA1/Snez0dexhzrFt6m510Hk3a6d1ycFCeRpfN06erf5w UttjkdMcsbikaVIpn5zB+qpVK+3vs+n0gJ68 X-Google-Smtp-Source: AGRyM1tNe7vUmw6FqW6dypY6pK6t8ejlIA8CMUbZIKq5phh+9YFIhT6aLwv3tcuv0VuBIzZUMqi1Vg== X-Received: by 2002:a63:803:0:b0:40d:dfde:a822 with SMTP id 3-20020a630803000000b0040ddfdea822mr2451326pgi.93.1656498629886; Wed, 29 Jun 2022 03:30:29 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:29 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:15 +0000 Message-Id: <20220629103018.4025635-7-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/9] ipu3: Configure imgu1 when necessary X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch configure and setup imgu1 when still capture and other streams are enabled. If only still capture is enabled, imgu0 is still being used with IPU3PipeModeVideo (i.e. VideoSnapshot) to ensure 3A is working. Still capture stream will be enabled in the following patches. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 173 +++++++++++++++++++++++---- 1 file changed, 149 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a13fb881..ec9d14d1 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -55,9 +55,11 @@ public: int loadIPA(); + void tryReturnBuffer(FrameBuffer *buffer); void imguOutputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); + void captureParamBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); void queuePendingRequests(); void cancelPendingRequests(); @@ -93,6 +95,8 @@ private: void paramsBufferReady(unsigned int id); void setSensorControls(unsigned int id, const ControlList &sensorControls, const ControlList &lensControls); + + std::map bufferReturnCounters; }; class IPU3CameraConfiguration : public CameraConfiguration @@ -106,7 +110,8 @@ public: Status validate() override; const StreamConfiguration &cio2Format() const { return cio2Configuration_; } - const ImgUDevice::PipeConfig imguConfig() const { return pipeConfig_; } + const ImgUDevice::PipeConfig imguConfig0() const { return pipeConfig0_; } + const ImgUDevice::PipeConfig imguConfig1() const { return pipeConfig1_; } /* Cache the combinedTransform_ that will be applied to the sensor */ Transform combinedTransform_; @@ -120,7 +125,8 @@ private: const IPU3CameraData *data_; StreamConfiguration cio2Configuration_; - ImgUDevice::PipeConfig pipeConfig_; + ImgUDevice::PipeConfig pipeConfig0_; + ImgUDevice::PipeConfig pipeConfig1_; }; class PipelineHandlerIPU3 : public PipelineHandler @@ -168,6 +174,8 @@ private: MediaDevice *cio2MediaDev_; MediaDevice *imguMediaDev_; + bool useImgu1_ = false; + Camera *inUseCamera_ = nullptr; std::vector ipaBuffers_; @@ -408,8 +416,8 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); - if (pipeConfig_.isNull()) { + pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); + if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; return Invalid; @@ -519,8 +527,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) IPU3CameraData *data = cameraData(camera); Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; + Stream *outCaptureStream = &data->outCaptureStream_; CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); + useImgu1_ = !imguConfig1.isNull(); + int ret; /* @@ -565,6 +578,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.enableLinks(true); + if (ret) + return ret; + } + /* * Pass the requested stream size to the CIO2 unit and get back the * adjusted format to be propagated to the ImgU output devices. @@ -607,14 +626,20 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream has been requested: return here to skip the ImgU configuration * part. */ - ImgUDevice::PipeConfig imguConfig = config->imguConfig(); - if (imguConfig.isNull()) + ImgUDevice::PipeConfig imguConfig0 = config->imguConfig0(); + if (imguConfig0.isNull()) return 0; - ret = imgu0_.configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig0, &cio2Format); if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.configure(imguConfig1, &cio2Format); + if (ret) + return ret; + } + /* Apply the format to the configured streams output devices. */ StreamConfiguration *mainCfg = nullptr; StreamConfiguration *vfCfg = nullptr; @@ -633,6 +658,15 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; + } else if (stream == outCaptureStream) { + ASSERT(useImgu1_); + ret = imgu1_.configureOutput(cfg, &outputFormat); + if (ret) + return ret; + + ret = imgu1_.configureViewfinder(cfg, &outputFormat); + if (ret) + return ret; } } @@ -648,22 +682,34 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu0_.imgu_->controls()); + ControlList ctrls0(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. - * - * \todo Figure out what the 'Still Capture' mode is meant for, and use - * it accordingly. */ - ctrls.set(V4L2_CID_IPU3_PIPE_MODE, - static_cast(IPU3PipeModeVideo)); - ret = imgu0_.imgu_->setControls(&ctrls); + ctrls0.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeVideo)); + ret = imgu0_.imgu_->setControls(&ctrls0); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; } + if (useImgu1_) { + ControlList ctrls1(imgu1_.imgu_->controls()); + /* + * \todo Figure out what the 'Still Capture' mode is meant for, + * and use it accordingly. + */ + ctrls1.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeStillCapture)); + ret = imgu1_.imgu_->setControls(&ctrls1); + if (ret) { + LOG(IPU3, Error) << "Unable to set pipe_mode control"; + return ret; + } + } + ipa::ipu3::IPAConfigInfo configInfo; configInfo.sensorControls = data->cio2_.sensor()->controls(); @@ -672,8 +718,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.lensControls = lens->controls(); configInfo.sensorInfo = sensorInfo; - configInfo.bdsOutputSize = config->imguConfig().bds; - configInfo.iif = config->imguConfig().iif; + configInfo.bdsOutputSize = config->imguConfig0().bds; + configInfo.iif = config->imguConfig0().iif; ret = data->ipa_->configure(configInfo, &data->ipaControls_); if (ret) { @@ -793,8 +839,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis CIO2Device *cio2 = &data->cio2_; int ret; - imgu0_.input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); + imgu0_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); imgu0_.output_->bufferReady.connect(data, &IPU3CameraData::imguOutputBufferReady); imgu0_.viewfinder_->bufferReady.connect(data, @@ -803,6 +849,16 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis &IPU3CameraData::paramBufferReady); imgu0_.stat_->bufferReady.connect(data, &IPU3CameraData::statBufferReady); + + if (useImgu1_) { + imgu1_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); + imgu1_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu1_.param_->bufferReady.connect(data, + &IPU3CameraData::captureParamBufferReady); + } + /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -834,10 +890,17 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; + if (useImgu1_) { + ret = imgu1_.start(); + if (ret) + goto error; + } + return 0; error: imgu0_.stop(); + imgu1_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); @@ -848,6 +911,11 @@ error: imgu0_.viewfinder_->bufferReady.disconnect(); imgu0_.param_->bufferReady.disconnect(); imgu0_.stat_->bufferReady.disconnect(); + imgu1_.input_->bufferReady.disconnect(); + imgu1_.output_->bufferReady.disconnect(); + imgu1_.param_->bufferReady.disconnect(); + + useImgu1_ = false; inUseCamera_ = nullptr; return ret; @@ -863,6 +931,7 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); ret |= imgu0_.stop(); + ret |= imgu1_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); @@ -874,7 +943,11 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->imgu0_->viewfinder_->bufferReady.disconnect(); data->imgu0_->param_->bufferReady.disconnect(); data->imgu0_->stat_->bufferReady.disconnect(); + data->imgu1_->input_->bufferReady.disconnect(); + data->imgu1_->output_->bufferReady.disconnect(); + data->imgu1_->param_->bufferReady.disconnect(); + useImgu1_ = false; inUseCamera_ = nullptr; } @@ -1320,22 +1393,45 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + bool hasYuv = false; + bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ for (auto it : info->request->buffers()) { const Stream *stream = it.first; FrameBuffer *outbuffer = it.second; - if (stream == &outStream_) + if (stream == &outStream_) { + hasYuv = true; imgu0_->output_->queueBuffer(outbuffer); - else if (stream == &vfStream_) + } else if (stream == &vfStream_) { + hasYuv = true; imgu0_->viewfinder_->queueBuffer(outbuffer); + } else if (stream == &outCaptureStream_) { + hasCapture = true; + + imgu1_->output_->queueBuffer(outbuffer); + } } - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + if (hasYuv) { + bufferReturnCounters[info->rawBuffer] += 1; - info->captureParamDequeued = true; + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } else { + info->paramDequeued = true; + info->metadataProcessed = true; + } + + if (hasCapture) { + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } else { + info->captureParamDequeued = true; + } } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) @@ -1356,6 +1452,15 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) * Buffer Ready slots */ +void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) +{ + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + cio2_.tryReturnBuffer(buffer); +} + /** * \brief Handle buffers completion at the ImgU output * \param[in] buffer The completed buffer @@ -1448,6 +1553,26 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) pipe()->completeRequest(request); } +void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) +{ + IPU3Frames::Info *info = frameInfos_.find(buffer); + if (!info) + return; + + info->captureParamDequeued = true; + + /* + * tryComplete() will delete info if it completes the IPU3Frame. + * In that event, we must have obtained the Request before hand. + * + * \todo Improve the FrameInfo API to avoid this type of issue + */ + Request *request = info->request; + + if (frameInfos_.tryComplete(info)) + pipe()->completeRequest(request); +} + void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { IPU3Frames::Info *info = frameInfos_.find(buffer); From patchwork Wed Jun 29 10:30:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16433 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26D33BE173 for ; Wed, 29 Jun 2022 10:30:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CF84F6564F; Wed, 29 Jun 2022 12:30:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498638; bh=7KqNgvMj8FSraVhSOVEy66P80+zobT6EsrmjhpASRWM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FPlfBELtbwKp7MaRNAQeaIdzFQoTTUYirUjuzlsj+forNUCkLUSkdHde+APSlTaw0 9+RkxNj0aNrCYlLQdKOdFetOfqbzMiiF91pJFDcCjlseBV5reuFajm0uJh9hTW1Pyb DmUsXjNsSNOj2IhD/fxPsiG4dcE9BRPNn2okL8hZWrnNFRzQs8ibGbYJCOAGKE8ZWg X7+sq+N68xnUMiN9ZlomBfLkmiCKQC91J+yiJZc+LWDnLolUiI3kKXsu3hTXBwF3L1 vvKKN+kvZkvtG7Go1gCE0CdfCWFoR4fUTID7JDXjhkkizheio/p8EejlzaYnLswTBC Z+2tncWHINTnw== Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AB2565635 for ; Wed, 29 Jun 2022 12:30:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="B4R3dTX2"; dkim-atps=neutral Received: by mail-pf1-x431.google.com with SMTP id k9so5470397pfg.5 for ; Wed, 29 Jun 2022 03:30:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w8WHyL8X8fhfTwZq5npRafRg2jbSksvdOtNL7mzbwao=; b=B4R3dTX26CHtDR5Xa/DGfx199aNyvuG4yspDHzjhql43P2+qaoJkuCr9mqrmbbp+jd iMYyjNxO1riqpeMm9Mp1ZzQ58Sw8FCNYJzBee6G7EgJQPVbOva6VnbdQuUPOzsDefGuA qlaRd41Jv3AMD7xB5pbWVUcWrsY26elqULcF8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w8WHyL8X8fhfTwZq5npRafRg2jbSksvdOtNL7mzbwao=; b=7i3w8UXGvnsjpiZE4U9HBQm7cNwlFazHPtDXnLQCsoJWoVvAReuK5fn7RXRTW3NehD jg39NNmf9+25DnjYuPrs/rW8OiU7lH+x6/9WH0hD70rFx1yaS1F/quJ+4roqnHkGzz0T r91WByaVGT5dkXjPJ2t1NeJVMGwTfPXOxnSDJ+z+69Fr5JO40g9eU9kYT/LhJd5J6oQD 95evwRVVMrWoHTP21NFK6FlzwdZMhf7zmUDT4JesSN71nywpdWRXpTHp7g7tni2VenBn wn8uQVprZpLGgc+IWYL91674ARroiI6AZhKGCXWQmS7UfZTVxyPBMxPmE55BtJ//C6dK EmtA== X-Gm-Message-State: AJIora9JJh9a+e/GeKETZAc6QdraD6Ru470jAG2/dRZMyRnTfruVc9f8 2nk+JNHM/2D5biYe5GTMcvGW9X+M8q53w3a0 X-Google-Smtp-Source: AGRyM1v26BGDV8+U4CZDaH4znjvOMv9ScM/Uv7W98WJxNLWJf+pCIm0mxpYIZtThvE9JIEx7uLVU3Q== X-Received: by 2002:a63:924f:0:b0:40c:f2d4:27e8 with SMTP id s15-20020a63924f000000b0040cf2d427e8mr2479173pgn.255.1656498631271; Wed, 29 Jun 2022 03:30:31 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:30 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:16 +0000 Message-Id: <20220629103018.4025635-8-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 7/9] ipu3: Assign |outCaptureStream| to StillCapture configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When StillCapture and other non-raw configurations are requested, assigns |outCaptureStream| instead of |outStream| to the StillCapture configuration. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 50 +++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index ec9d14d1..e26c2736 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -254,8 +254,10 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * \todo Clarify the IF and BDS margins requirements. */ unsigned int rawCount = 0; - unsigned int yuvCount = 0; + unsigned int videoCount = 0; + unsigned int stillCount = 0; Size maxYuvSize; + Size maxVideoSize; Size rawSize; for (const StreamConfiguration &cfg : config_) { @@ -264,16 +266,29 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { rawCount++; rawSize.expandTo(cfg.size); + } else if (cfg.streamRole == StreamRole::StillCapture) { + if (stillCount != 0) + return Invalid; + + stillCount++; + maxYuvSize.expandTo(cfg.size); } else { - yuvCount++; + videoCount++; maxYuvSize.expandTo(cfg.size); + maxVideoSize.expandTo(cfg.size); } } - if (rawCount > 1 || yuvCount > 2) { + if (videoCount == 0 && stillCount == 1) { + stillCount = 0; + videoCount = 1; + maxVideoSize.expandTo(maxYuvSize); + } + + if (rawCount > 1 || videoCount > 2) { LOG(IPU3, Debug) << "Camera configuration not supported"; return Invalid; - } else if (rawCount && !yuvCount) { + } else if (rawCount && !stillCount && !videoCount) { /* * Disallow raw-only camera configuration. Currently, ImgU does * not get configured for raw-only streams and has early return @@ -316,6 +331,9 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() ImgUDevice::Pipe pipe{}; pipe.input = cio2Configuration_.size; + ImgUDevice::Pipe pipe1{}; + pipe1.input = cio2Configuration_.size; + /* * Adjust the configurations if needed and assign streams while * iterating them. @@ -380,18 +398,34 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() cfg->stride = info.stride(cfg->size.width, 0, 1); cfg->frameSize = info.frameSize(cfg->size, 1); + if (stillCount == 1 && cfg->streamRole == StreamRole::StillCapture) { + LOG(IPU3, Debug) << "Assigned " + << cfg->toString() + << " to the imgu1 main output"; + cfg->setStream(const_cast(&data_->outCaptureStream_)); + + pipe1.main = cfg->size; + pipe1.viewfinder = pipe1.main; + + pipeConfig1_ = data_->imgu1_->calculatePipeConfig(&pipe1); + if (pipeConfig1_.isNull()) { + LOG(IPU3, Error) << "Failed to calculate pipe configuration: " + << "unsupported resolutions."; + return Invalid; + } /* * Use the main output stream in case only one stream is * requested or if the current configuration is the one * with the maximum YUV output size. */ - if (mainOutputAvailable && - (originalCfg.size == maxYuvSize || yuvCount == 1)) { + } else if (mainOutputAvailable && + (originalCfg.size == maxVideoSize || + videoCount == 1)) { cfg->setStream(const_cast(&data_->outStream_)); mainOutputAvailable = false; pipe.main = cfg->size; - if (yuvCount == 1) + if (videoCount == 1) pipe.viewfinder = pipe.main; LOG(IPU3, Debug) << "Assigned " << cfg->toString() @@ -415,7 +449,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() } /* Only compute the ImgU configuration if a YUV stream has been requested. */ - if (yuvCount) { + if (videoCount) { pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " From patchwork Wed Jun 29 10:30:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16434 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1BF0ABE173 for ; Wed, 29 Jun 2022 10:30:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B459A65646; Wed, 29 Jun 2022 12:30:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498639; bh=FbJsGszTvkjsFHZa24XQPE191prSV0m7J1vk45/E0Dg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MnnjLoJSyjhcH6ldf23SQ7WNBDl9idaHlh4JDwvaX6n+R3eemic7L5zX1L4cjh7wT ttF5x+sASkaziscfKtp+ksoQTqZvNCdn7q5alz0JWVCCfSmU4VMCqjmT7LAf+BNC+P GJtSgvkg0aY1NPM2OtkA25ulGOGHT6nspwwjp1kJlNqg+bCcx0Dz0l6QCqkYN173Bo 1A0qMY9LY9B8CUnXl/smTkPFubnohUiyicL2c76YxISpJnU/1/sy6sysbXrJHZ1hEa C3pGKeDNPaV9M16HfEbXDwfN80eB+MZLJ70zgqDrMPgB8/0DZyo/MH/4XOvOuhtzGx EOGnmVLreEy+A== Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 29F7665635 for ; Wed, 29 Jun 2022 12:30:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CfFfAmYm"; dkim-atps=neutral Received: by mail-pf1-x433.google.com with SMTP id 136so9576186pfy.10 for ; Wed, 29 Jun 2022 03:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=umd9+EQTp74CNkn5SYDVVvbPaWnB1mCRZape6APHeiY=; b=CfFfAmYmg7gc/gNFOasafoRl6iqspMH+pSxOdq97vt5UL9rW9xsvshzqQWlikvFjF3 mnXFJ5lF7iaNFSorCCC9rypUzQKrlu3oy4qGd9EltPFtwfx9Bcp4RhHnNJAdk1+9wy9I 9b7zyurJJ+rkJ4Bld58ouI1gvHjmRKCL8VBww= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=umd9+EQTp74CNkn5SYDVVvbPaWnB1mCRZape6APHeiY=; b=Ur+VGzUgWqPd5YDAzBwXIToybhFqmHJzZmcPo9EwwjD2BAYFYPZqW81svug+p+0wMz +3t7jKI+HOMqFqeE0z/SLN7jsrj0JLPSEcK5yD2/B2xT8lcX0pY64YKhFoypbTu5Jd+w GXunnxbZYUtV+v/wbvBPHTpNLiLdoaLX7beoOrN2Av5VQ+TevHEv3L3zoE3xldoujrZU zWCeCwtbDkVlpBU+Dwg7Ff3rq5/okVnoGj5n6YqhbomZJOt7jM8F+5eWtYzIF0WQdcZF aQx2gXtW1FywlrAGbl2LaLUP0EhZxLn0fvSY8e4o4JNsA06cHmYSPVUE6dnss++NsUan NnxA== X-Gm-Message-State: AJIora8xNWkuZuRTtq7l+UL0FF3lGuED+gunP1ASCu9Ie/sWZKvZB+k4 cirzBbqEpWvt9a+BaQROez5dnSMV3RoNlz09 X-Google-Smtp-Source: AGRyM1uUDQYDNm/ktMgxN8iBtRwfTpPggwv/J/lMmL0bW8K7lw3+XIs74UzGC4Dr+yC3/LjJZvpRaQ== X-Received: by 2002:a63:5810:0:b0:40d:77fb:1c25 with SMTP id m16-20020a635810000000b0040d77fb1c25mr2415856pgb.570.1656498632456; Wed, 29 Jun 2022 03:30:32 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:32 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:17 +0000 Message-Id: <20220629103018.4025635-9-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/9] Request StillCapture stream as default in Android adapter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the pipeline handler accepts, the Android adapter requests to handle the StillCapture stream separately, instead of being mapped to an existing video/preview stream or creating a new video stream. Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 235 +++++++++++++++++++--------------- src/android/camera_device.h | 21 +++ 2 files changed, 154 insertions(+), 102 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..bbb72615 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -36,23 +36,6 @@ LOG_DECLARE_CATEGORY(HAL) namespace { -/* - * \struct Camera3StreamConfig - * \brief Data to store StreamConfiguration associated with camera3_stream(s) - * \var streams List of the pairs of a stream requested by Android HAL client - * and CameraStream::Type associated with the stream - * \var config StreamConfiguration for streams - */ -struct Camera3StreamConfig { - struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; - }; - - std::vector streams; - StreamConfiguration config; -}; - /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. @@ -536,16 +519,6 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return -EINVAL; #endif - /* - * Generate an empty configuration, and construct a StreamConfiguration - * for each camera3_stream to add to it. - */ - std::unique_ptr config = camera_->generateConfiguration(); - if (!config) { - LOG(HAL, Error) << "Failed to generate camera configuration"; - return -EINVAL; - } - /* * Clear and remove any existing configuration from previous calls, and * ensure the required entries are available without further @@ -614,93 +587,102 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } + std::unique_ptr config; + /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { - CameraStream::Type type; - int index = -1; - - /* Search for a compatible stream in the non-JPEG ones. */ - for (size_t i = 0; i < streamConfigs.size(); ++i) { - Camera3StreamConfig &streamConfig = streamConfigs[i]; - const auto &cfg = streamConfig.config; - - /* - * \todo The PixelFormat must also be compatible with - * the encoder. - */ - if (cfg.size.width != jpegStream->width || - cfg.size.height != jpegStream->height) - continue; - - LOG(HAL, Info) - << "Android JPEG stream mapped to libcamera stream " << i; - - type = CameraStream::Type::Mapped; - index = i; - - /* - * The source stream will be read by software to - * produce the JPEG stream. - */ - camera3_stream_t *stream = streamConfig.streams[0].stream; - stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; - break; - } + /* This stream will be produced by hardware. */ + jpegStream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; /* - * Without a compatible match for JPEG encoding we must - * introduce a new stream to satisfy the request requirements. + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. */ - if (index < 0) { + Camera3StreamConfig sConfig; + sConfig.config.streamRole = libcamera::StreamRole::StillCapture; + sConfig.config.size.width = jpegStream->width; + sConfig.config.size.height = jpegStream->height; + sConfig.config.pixelFormat = formats::NV12; + + std::vector streamConfigsCopy(streamConfigs); + streamConfigsCopy.push_back(std::move(sConfig)); + + LOG(HAL, Info) << "Adding " << sConfig.config.toString() + << " for MJPEG support"; + + streamConfigsCopy.back().streams.push_back({ jpegStream, CameraStream::Type::Internal }); + + config = tryValidation(streamConfigsCopy, jpegStream); + if (config) { + streamConfigs = streamConfigsCopy; + } else { + /* The JPEG stream will be produced by software. */ + jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; + + CameraStream::Type type; + int index = -1; + /* Search for a compatible stream in the non-JPEG ones. */ + for (size_t i = 0; i < streamConfigs.size(); ++i) { + Camera3StreamConfig &streamConfig = streamConfigs[i]; + const auto &cfg = streamConfig.config; + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size.width != jpegStream->width || + cfg.size.height != jpegStream->height) + continue; + + LOG(HAL, Info) + << "Android JPEG stream mapped to libcamera stream " << i; + index = i; + type = CameraStream::Type::Mapped; + + /* + * The source stream will be read by software to + * produce the JPEG stream. + */ + camera3_stream_t *stream = streamConfig.streams[0].stream; + stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + break; + } + /* - * \todo The pixelFormat should be a 'best-fit' choice - * and may require a validation cycle. This is not yet - * handled, and should be considered as part of any - * stream configuration reworks. + * Without a compatible match for JPEG encoding we must + * introduce a new stream to satisfy the request requirements. */ - Camera3StreamConfig streamConfig; - streamConfig.config.size.width = jpegStream->width; - streamConfig.config.size.height = jpegStream->height; - streamConfig.config.pixelFormat = formats::NV12; - streamConfigs.push_back(std::move(streamConfig)); - - LOG(HAL, Info) << "Adding " << streamConfig.config.toString() - << " for MJPEG support"; - - type = CameraStream::Type::Internal; - index = streamConfigs.size() - 1; - } - - /* The JPEG stream will be produced by software. */ - jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; - - streamConfigs[index].streams.push_back({ jpegStream, type }); - } - - sortCamera3StreamConfigs(streamConfigs, jpegStream); - for (const auto &streamConfig : streamConfigs) { - config->addConfiguration(streamConfig.config); + if (index < 0) { + /* + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. + */ + Camera3StreamConfig streamConfig; + streamConfig.config.size.width = jpegStream->width; + streamConfig.config.size.height = jpegStream->height; + streamConfig.config.pixelFormat = formats::NV12; + streamConfigs.push_back(std::move(streamConfig)); + + LOG(HAL, Info) << "Adding " << streamConfig.config.toString() + << " for MJPEG support"; + + type = CameraStream::Type::Internal; + index = streamConfigs.size() - 1; + } - for (auto &stream : streamConfig.streams) { - streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); - stream.stream->priv = static_cast(&streams_.back()); + streamConfigs[index].streams.push_back({ jpegStream, type }); } } - switch (config->validate()) { - case CameraConfiguration::Valid: - break; - case CameraConfiguration::Adjusted: - LOG(HAL, Info) << "Camera configuration adjusted"; - - for (const StreamConfiguration &cfg : *config) - LOG(HAL, Info) << " - " << cfg.toString(); + if (!config) { + config = tryValidation(streamConfigs, jpegStream); + if (!config) + return -EINVAL; - return -EINVAL; - case CameraConfiguration::Invalid: - LOG(HAL, Info) << "Camera configuration invalid"; - return -EINVAL; } /* @@ -1527,3 +1509,52 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons return resultMetadata; } + +std::unique_ptr CameraDevice::tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream) +{ + /* + * Generate an empty configuration, and construct a StreamConfiguration + * for each camera3_stream to add to it. + */ + std::unique_ptr config = camera_->generateConfiguration(); + if (!config) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + return nullptr; + } + + sortCamera3StreamConfigs(streamConfigs, jpegStream); + for (const auto &streamConfig : streamConfigs) { + config->addConfiguration(streamConfig.config); + + for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, + stream.stream, config->size() - 1); + stream.stream->priv = static_cast(&streams_.back()); + } + } + + switch (config->validate()) { + case CameraConfiguration::Valid: + break; + case CameraConfiguration::Adjusted: + LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config) + LOG(HAL, Info) << " - " << cfg.toString(); + + config.reset(); + break; + case CameraConfiguration::Invalid: + LOG(HAL, Info) << "Camera configuration invalid"; + + config.reset(); + break; + } + + if (!config) + streams_.clear(); + + return config; +} diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..8540e3f5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -34,6 +34,23 @@ class Camera3RequestDescriptor; struct CameraConfigData; +/* + * \struct Camera3StreamConfig + * \brief Data to store StreamConfiguration associated with camera3_stream(s) + * \var streams List of the pairs of a stream requested by Android HAL client + * and CameraStream::Type associated with the stream + * \var config StreamConfiguration for streams + */ +struct Camera3StreamConfig { + struct Camera3Stream { + camera3_stream_t *stream; + CameraStream::Type type; + }; + + std::vector streams; + libcamera::StreamConfiguration config; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -101,6 +118,10 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + std::unique_ptr tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream); + unsigned int id_; camera3_device_t camera3Device_; From patchwork Wed Jun 29 10:30:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16435 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CEF2BE173 for ; Wed, 29 Jun 2022 10:30:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BCEE6564D; Wed, 29 Jun 2022 12:30:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498640; bh=ly5Y8KqbL8yfCBnsfqyc9LvXVdMjapsBLtxS538uMHw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MG1CzcphGpH0x0M61u9hmD67hzWEfTfpzVltUT+kyf6zfOrNu5psUZQY4+5qu74kJ bojF75iA/LlDqS20099yEVRSo9/5WY4k7Q5Vs0AKeojrEw+lfspjOxkDtGeoc1CWua 0g1qdzwTX+wW39N0XN1i6OIAXGZzCQW8XdwFGDeTq3Jif+8cZP7/2B2IcjmsGzSWtI NXd0fgLGUR5me7NmCYuWzAAMOos7XemWSuxi5Zun2z8lvdXh4PbH9klt7N8VX1OHn3 /4XhoVhRIRidzx1noPet0effrjJsVS9LJCsDoebw8zKWinXyXJkYBv4bhpUOkf6aHF HPRnOwzbYJyXg== Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E71365643 for ; Wed, 29 Jun 2022 12:30:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="dWaUQsLs"; dkim-atps=neutral Received: by mail-pl1-x631.google.com with SMTP id q18so13664048pld.13 for ; Wed, 29 Jun 2022 03:30:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=; b=dWaUQsLszzcOWnzt4bRlCFGgnNggIF7mL9rNnjkAdXFREg6hyPxMjaMW5d6+0hu6mp YfgvQ0IXwUdlsrzi2K+BTPnFMG/+ODUC4UbCE92Ec2kiM2Lv7rzHirDLFPoJE6SSmxob U8lxwDmaqpPDEPRIZUZ6X2HPbkKAAY5IBavt8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=; b=rN5UpK7xDfHCWSaedg3UrIBNhBx9o6nSgMtHhBEiSMJI6ik6V68m6I8ypa7wN7Rhlx Be4DetXxiK8yOtINtqPHEV3zHEOty1rM+o8rDs/x0fyzxuFsEeM0XLif9DtVaaKCnF7j CaX7IIWLRVM8oLyS/+Lp3dqgQKCbDjKHBlkXVbCyFk6uWyatLkdkaP752WEr7RG0xy93 nSUCvhn/5Mdl2B4Xh+fVh9rLDBY6EVVmpiQrVTu1yS/mqJPrOcVMN+mGYKsZ6R4SLzw4 pScZpv+MX4+N4EzyqbQFDF/H2bzitkxnc+9fKZ6XR0Im74mRNoOiovVz5KJF8zw6LmAh Fafg== X-Gm-Message-State: AJIora/+EMOHyZhvLDFsH/wOQ9WYpZXihFAWrS/EeoKyuWSkjJoViB6F pFXD7kdGIu3eHZJv0js5dJzBq5hNNP3OqZMH X-Google-Smtp-Source: AGRyM1tmby4XGwy9PJr1H6plJ0ln4JEmuyNRUCGag1nZ9rqeZLbcSk/8VAL1t6Oa2oQo8oexe8/f/A== X-Received: by 2002:a17:902:d50e:b0:16a:13d:30ab with SMTP id b14-20020a170902d50e00b0016a013d30abmr9819182plg.31.1656498633846; Wed, 29 Jun 2022 03:30:33 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:33 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:18 +0000 Message-Id: <20220629103018.4025635-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL, handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e26c2736..8689cf8b 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastRequestSequence_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + data->firstRequest_ = true; + data->lastRequestSequence_ = -1; + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); useImgu1_ = !imguConfig1.isNull(); @@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastRequestSequence_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;