From patchwork Thu May 26 08:27:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3837ABD161 for ; Thu, 26 May 2022 08:27:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C76696566B; Thu, 26 May 2022 10:27:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553671; bh=1K4XxfbF/A4NXjuuloZdN9XN+KSBjmVfu7DKtWqF/G0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=bG1f9t3/Qbkr7MahYuiZn4zALPOZnkcD/VPhFKeU3Dt4YoaYDmN7Ryj95gCXMdw0r a49axmKl7ICvHIg/QfuGPm3yIeiJ8fmNkZ0IoNRmBwODA4VvQD9hA99EQKWcx3WJps 7xovV8r+kYRSkaLH89pX8itPBpvaFMfixhzW5wm0t1Vs2STwIQML9drU5mPLdE64kj ieYTVDO2EhisYJ+3wHNykRT21UnGSHlvsVwd3GyumLgRswsbf4lh2Ec7lfGIUgNqa7 nmamRQomszP6XStP7cmto/puGCL+jDKvQotkdo3Rbz610jyK1Ce+cB5sVJmGC1fDiD uekALcIitxzjA== Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5963B65661 for ; Thu, 26 May 2022 10:27:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Y6B435tF"; dkim-atps=neutral Received: by mail-pj1-x102a.google.com with SMTP id o10-20020a17090a4e8a00b001df2fcdc165so3829890pjh.0 for ; Thu, 26 May 2022 01:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M+Ps0wwMRSHDTK6TtrIDjEssnLfjyE4zU0Op6qAgYBo=; b=Y6B435tFN9OVyZAKyLA9ro4z6wbgVfv7UfP/z9tC5OLT5ybtJ03nx1Zt8YPizORd/9 9HduTWD6my/rTJL6kkdLnYVv4uEmSuwPpMoBqGDOLTMO8CrHCg8ypoTmkmRd/wtnH+8I j2vETHXTRpiwsjGMLiapyulJIpqmsvENR7QmE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M+Ps0wwMRSHDTK6TtrIDjEssnLfjyE4zU0Op6qAgYBo=; b=4zw5o12R/Yrb3AchXycXsQxeeKrgKTB0kQAmrGwbsWPdJMEttoOgB03shAImC5Lyc4 pikBCfvKIZFjLMe1P52q5gXAQsVUGpzvJ+xHhmCK+ogN80auU3/NDLFk7AJ7kQwD5Dlp fdeu531nVUCwbm703XzS18cYTOfLicS5pHaSFE59m2MOnuz5MCAb/faN/vHcr0sFdEAm Mk/J3DdB4K2y88a5ssUX7lRZZgH7j6OY2yJlov2gre1jFBfVzXFCOdjl9/X0PT1z3ClU Ff63r9pii6tsNqSn/AMAG9G1alGK2+VtiAqr4xErHwAUhzq0im+wf9IDUHIm+qe7xzyb PURg== X-Gm-Message-State: AOAM5331Ybv50Q7GUNcYL9E2m67mmk6xMAO7fFAT4AG/k51dFSjg02hl RbIiEEO/2ILGXYwcMsXAPgpiE5J3eD9ui8cf X-Google-Smtp-Source: ABdhPJxoTv9g57ECCBk8LzfOXYHIdfK1ppu6fyKzhWcLzofACNujzJrgukSHwBU8prptcMk9HVcQrg== X-Received: by 2002:a17:903:22d1:b0:162:166:6a96 with SMTP id y17-20020a17090322d100b0016201666a96mr27330203plg.127.1653553666329; Thu, 26 May 2022 01:27:46 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:45 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:29 +0000 Message-Id: <20220526082737.1081262-2-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/9] Add StreamRole in StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang To let ipu3 pipeline handler understand the usage of streams, this patch adds StreamRole member variable in StreamConfiguration, which should be set by the (Android) adapter. Signed-off-by: Harvey Yang --- include/libcamera/stream.h | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index f0ae7e62..c9e79226 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -22,6 +22,15 @@ namespace libcamera { class Camera; class Stream; +enum StreamRole { + Raw, + StillCapture, + VideoRecording, + Viewfinder, +}; + +using StreamRoles = std::vector; + class StreamFormats { public: @@ -50,6 +59,8 @@ struct StreamConfiguration { std::optional colorSpace; + StreamRole streamRole; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } @@ -61,15 +72,6 @@ private: StreamFormats formats_; }; -enum StreamRole { - Raw, - StillCapture, - VideoRecording, - Viewfinder, -}; - -using StreamRoles = std::vector; - class Stream { public: From patchwork Thu May 26 08:27:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2A30C3256 for ; Thu, 26 May 2022 08:27:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B322B65667; Thu, 26 May 2022 10:27:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553672; bh=ABgfGQ/ElEe+i40usmKjEuyVAwo8ycwZk/ueid1QkgM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=b9jE1JjQeboIMNO0JPSLQp7RW1lcpqL3aabEVfxBSCL/3P/g/XoQLq0bC0y1FY5qM dR2del/f2mVL18hmBFADxwQ8v70mLiFKRMQVMj7SPHTWOjc45to59xf0Hrjo1NUiAu WdFGg/0wg/CoLsagfhVMa5zXOTc872EHokF2lD0WQjbdfp3LKxYnB9NwGjp9c/1rDu CuvM1CJdtwdpO1B9ZCpxJ15Yjap2Nn4eqGeQN0uPhsse5jICCbHxm/T+WXQ65RnsSS Hfe5RvUSVIrZHhmPNoOzRAHGD8zdm39nnJSCFSSJXlVoOSGK3fvX0TDweZZnI0OlMJ hRkaMqpYMH+dw== Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3B956565D for ; Thu, 26 May 2022 10:27:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="bD5sbJi7"; dkim-atps=neutral Received: by mail-pj1-x102b.google.com with SMTP id q92-20020a17090a17e500b001e0817e77f6so3794171pja.5 for ; Thu, 26 May 2022 01:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7I54rMUuKWDVxcfwPnpOucKfcZsO9xgIQyAJBV3cSkg=; b=bD5sbJi7RkPAsAQFopJH4ay0kCF/jrITjA7NGbp5GT0uJ61av5xtflEy4SUIwsmLDB MmWPT9zPQ7sbq4VCXrrZxatvL1vKtqkWQ1Af4JYXfpdBEU+yP6//tarmv+G2uUa/WtAM +wU8AvCEwgjQ/JBp739SkCY9NRw5MBNdqRutg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7I54rMUuKWDVxcfwPnpOucKfcZsO9xgIQyAJBV3cSkg=; b=mwV5le7wvFeNNxr7Q8ZJHedwYqeVtMGsrEMIhnPjvnB+Oh06hFs92xZFSRDAoRCXpZ ehKoquDuvbnLXG/1vZv67oy5rKLx4WcmCJBexGnm54UlYgPDotrK6X8NIviXAH2hzcvp zwBMp4JYZZF7Qg5hu0HDUt4JAE4GoYny0WEGyHvS4U4CaMFaHJjO3fqHCmUKWxQO0Xg2 Zkq2mh2VtW1gLxMKDZmpWDxyQFG6zTvrRtMCp3O8BcOoABYNe1iyFqjL480aAR8VRm19 fef6aBIetzyiLOLTPCbLZMRuEDqCpU1UZq7+cenzLh8IYLxgmd+vtDSp0v1rS0Fz0Kiz vDKg== X-Gm-Message-State: AOAM531okNrWc50NjntFY4BIGhia2TU5PY8iMcN+avZ3c1t3tSW+j6vY 0mlb0dzPv+jDz7xFsVeJguBHmwVL6vTrXgRi X-Google-Smtp-Source: ABdhPJxPbTlPe1zKARFtM+3NQx+s/mN/GqEzY9GZJMTtwvb5sP3LEv/mdXYkHQC21ZU9MrGcniqcjw== X-Received: by 2002:a17:90b:1095:b0:1e0:ccaa:e5f0 with SMTP id gj21-20020a17090b109500b001e0ccaae5f0mr1432742pjb.198.1653553667863; Thu, 26 May 2022 01:27:47 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:47 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:30 +0000 Message-Id: <20220526082737.1081262-3-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/9] ipu3: Allow only one camera being started X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang As we hardly have use cases/applications that need both cameras at the same time, this patch adds a rule that only one camera can be started one time. This also allows the following patches that use both imgus to process frames from one single camera. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index fd989e61..c943ee6a 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -166,6 +166,8 @@ private: MediaDevice *cio2MediaDev_; MediaDevice *imguMediaDev_; + Camera *inUseCamera_ = nullptr; + std::vector ipaBuffers_; }; @@ -765,6 +767,14 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { + /* + * Enforce that only a single camera can be used at a time to use both + * ImgUs on the camera, so that StillCapture stream can adopt another + * set of configuration. + */ + if (inUseCamera_ && inUseCamera_ != camera) + return -EBUSY; + IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; @@ -781,6 +791,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) return ret; + inUseCamera_ = camera; + ret = data->ipa_->start(); if (ret) goto error; @@ -808,6 +820,8 @@ error: freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); + inUseCamera_ = nullptr; + return ret; } @@ -826,6 +840,8 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + + inUseCamera_ = nullptr; } void IPU3CameraData::cancelPendingRequests() From patchwork Thu May 26 08:27:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16052 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3803DBD161 for ; Thu, 26 May 2022 08:27:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E28856566B; Thu, 26 May 2022 10:27:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553674; bh=ymXHoMyozBcIT7eofqpXqc4oG/a1mTpchS1CYhvalLM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=1y/JckfoDTjvc3bT/7F8Wm3YM0vV9g6Tq55d5pDtBYA492H6H4le45uQo1EhtDPDG tPbg7O1S5uLImHtXgWvuW9xmLz+NFqQEB7FRwbExtvhcWchXYLAEj6XcVvTzg+KxYI 4IyKWkFjngPy0Y0abJ2vFA2EhBMYZC1Ba7/Ss/oNaJx05pEls5XBiFDzrsb9DZ4VHH cjFmpz4Nc96tzdOOz7MDgYbbGOr9o03dSZlpi1RfhNeM/utn5k2Md7EYEifhdLCNtk z1Pss6Ri0k6+PRIvpSbQF3izFJPCSbw/oiiatXqlOskvgpTmqEIyQAgQ0fgu7W0NzQ fEsqIpjjtADLQ== Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 484A265667 for ; Thu, 26 May 2022 10:27:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="OOLvfqfH"; dkim-atps=neutral Received: by mail-pj1-x1029.google.com with SMTP id w2-20020a17090ac98200b001e0519fe5a8so1135484pjt.4 for ; Thu, 26 May 2022 01:27:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PXWKq4slraAOmwCM/dQggWwxkqw+wuUDxXVYBkvYVVk=; b=OOLvfqfH2c4tdpSAmU+bWG1R3hcuZecFKd+3f4mCUnViZBQLmqPlrLGzhoY6Ja7Dpp +D0mMre0C6nZsi/V3FsniPuRcAmW3MEf56KYjjr41LFPjdADv8yuAdQyvuNY737L3ySQ xufhQcNa9wmCI6t9X/w6+6cEShnlT9IuwXwIM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PXWKq4slraAOmwCM/dQggWwxkqw+wuUDxXVYBkvYVVk=; b=anZEnAmxtoXuVetgzJRYTE1YWNrAxqQgaMKuTwvS92H3yFocMuJDbJF7xYOgMtrZgL xQJZbihGX8tGKY5TfvAMs2wAUy5/aUMeJdfqt50IXtjZBnCyIjrzyTwOLiul7gtWhyGi OqgKPFaZdBB3ns4o0a7qVSXj9o5cPW4raSATl8PVrFPqbqIiEXpWKdYHlqwv0j+7ltgb PHEK1kIKUFwhbHv2DSbLbf/y69fpAH0wui1KUfnkGHfgZdqNBY40/B47GS0wjzJscKSf RiCoDpVHgStlAy8/noBPw4m8Njq2/Xqqm60PQCmez4fXkUZXjx7uwzBxEQRf/7NBzyXX h4vw== X-Gm-Message-State: AOAM530EqxbNUnGideKqlgB3j96ZCqNJpkKy6y60kK36Fuc8NqGQ1Xgf NN+BcTJUJXfGMaedlQDVITTpW34ZCwNBjVXl X-Google-Smtp-Source: ABdhPJxz5BtlHZPBpEReONPYTBguHQw0QBqcD7TCJahXhC/ZDfD7nrigBCI5vmELznMEpmFmGwAZtw== X-Received: by 2002:a17:90a:1506:b0:1e2:5e03:819f with SMTP id l6-20020a17090a150600b001e25e03819fmr414516pja.122.1653553669452; Thu, 26 May 2022 01:27:49 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:48 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:31 +0000 Message-Id: <20220526082737.1081262-4-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/9] ipu3: Use imgu0 as default X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang With only one camera being started, we can always use imgu0 to process frames (for video/preview). In the following patches, we'll use imgu1 for still capture if needed. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 86 ++++++++++++++++------------ 1 file changed, 48 insertions(+), 38 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index c943ee6a..e219f704 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -64,7 +64,8 @@ public: void frameStart(uint32_t sequence); CIO2Device cio2_; - ImgUDevice *imgu_; + ImgUDevice *imgu0_; + ImgUDevice *imgu1_; Stream outStream_; Stream vfStream_; @@ -406,7 +407,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu_->calculatePipeConfig(&pipe); + pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; @@ -518,7 +519,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; V4L2DeviceFormat outputFormat; int ret; @@ -560,7 +560,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream which is for raw capture, in which case no buffers will * ever be queued to the ImgU. */ - ret = data->imgu_->enableLinks(true); + ret = imgu0_.enableLinks(true); if (ret) return ret; @@ -610,7 +610,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (imguConfig.isNull()) return 0; - ret = imgu->configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig, &cio2Format); if (ret) return ret; @@ -624,12 +624,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (stream == outStream) { mainCfg = &cfg; - ret = imgu->configureOutput(cfg, &outputFormat); + ret = imgu0_.configureOutput(cfg, &outputFormat); if (ret) return ret; } else if (stream == vfStream) { vfCfg = &cfg; - ret = imgu->configureViewfinder(cfg, &outputFormat); + ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; } @@ -641,13 +641,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * be at least one active stream in the configuration request). */ if (!vfCfg) { - ret = imgu->configureViewfinder(*mainCfg, &outputFormat); + ret = imgu0_.configureViewfinder(*mainCfg, &outputFormat); if (ret) return ret; } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu->imgu_->controls()); + ControlList ctrls(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. @@ -657,7 +657,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) */ ctrls.set(V4L2_CID_IPU3_PIPE_MODE, static_cast(IPU3PipeModeVideo)); - ret = imgu->imgu_->setControls(&ctrls); + ret = imgu0_.imgu_->setControls(&ctrls); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; @@ -691,9 +691,9 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, unsigned int count = stream->configuration().bufferCount; if (stream == &data->outStream_) - return data->imgu_->output_->exportBuffers(count, buffers); + return imgu0_.output_->exportBuffers(count, buffers); else if (stream == &data->vfStream_) - return data->imgu_->viewfinder_->exportBuffers(count, buffers); + return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); @@ -711,7 +711,6 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, int PipelineHandlerIPU3::allocateBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - ImgUDevice *imgu = data->imgu_; unsigned int bufferCount; int ret; @@ -721,26 +720,26 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->rawStream_.configuration().bufferCount, }); - ret = imgu->allocateBuffers(bufferCount); + ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; - for (const std::unique_ptr &buffer : imgu->paramBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.paramBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } - for (const std::unique_ptr &buffer : imgu->statBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.statBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -760,7 +759,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) data->ipa_->unmapBuffers(ids); ipaBuffers_.clear(); - data->imgu_->freeBuffers(); + imgu0_.freeBuffers(); return 0; } @@ -777,9 +776,18 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; int ret; + imgu0_.input_->bufferReady.connect(&data->cio2_, + &CIO2Device::tryReturnBuffer); + imgu0_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.viewfinder_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.param_->bufferReady.connect(data, + &IPU3CameraData::paramBufferReady); + imgu0_.stat_->bufferReady.connect(data, + &IPU3CameraData::statBufferReady); /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -807,19 +815,24 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; - ret = imgu->start(); + ret = imgu0_.start(); if (ret) goto error; return 0; error: - imgu->stop(); + imgu0_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); + imgu0_.input_->bufferReady.disconnect(); + imgu0_.output_->bufferReady.disconnect(); + imgu0_.viewfinder_->bufferReady.disconnect(); + imgu0_.param_->bufferReady.disconnect(); + imgu0_.stat_->bufferReady.disconnect(); inUseCamera_ = nullptr; return ret; @@ -834,13 +847,19 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); - ret |= data->imgu_->stop(); + ret |= imgu0_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + data->imgu0_->input_->bufferReady.disconnect(); + data->imgu0_->output_->bufferReady.disconnect(); + data->imgu0_->viewfinder_->bufferReady.disconnect(); + data->imgu0_->param_->bufferReady.disconnect(); + data->imgu0_->stat_->bufferReady.disconnect(); + inUseCamera_ = nullptr; } @@ -1184,7 +1203,8 @@ int PipelineHandlerIPU3::registerCameras() * only, and assign imgu0 to the first one and imgu1 to the * second. */ - data->imgu_ = numCameras ? &imgu1_ : &imgu0_; + data->imgu0_ = &imgu0_; + data->imgu1_ = &imgu1_; /* * Connect video devices' 'bufferReady' signals to their @@ -1198,16 +1218,6 @@ int PipelineHandlerIPU3::registerCameras() &IPU3CameraData::cio2BufferReady); data->cio2_.bufferAvailable.connect( data.get(), &IPU3CameraData::queuePendingRequests); - data->imgu_->input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); - data->imgu_->output_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->viewfinder_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->param_->bufferReady.connect(data.get(), - &IPU3CameraData::paramBufferReady); - data->imgu_->stat_->bufferReady.connect(data.get(), - &IPU3CameraData::statBufferReady); /* Create and register the Camera instance. */ const std::string &cameraId = cio2->sensor()->id(); @@ -1300,14 +1310,14 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) FrameBuffer *outbuffer = it.second; if (stream == &outStream_) - imgu_->output_->queueBuffer(outbuffer); + imgu0_->output_->queueBuffer(outbuffer); else if (stream == &vfStream_) - imgu_->viewfinder_->queueBuffer(outbuffer); + imgu0_->viewfinder_->queueBuffer(outbuffer); } - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Thu May 26 08:27:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 21E67C3256 for ; Thu, 26 May 2022 08:27:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFC8665664; Thu, 26 May 2022 10:27:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553675; bh=RvPnM55UMEnIawbbO2uXtEUTa/XQndiwHl9Bn23kq1I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=NxfvnEL1GzdvC8z3Rnp4cLk3sVsfuWsOpyqlk0sB8qCncOc4nGzisjwrg51rMHEMW KxUNC8grwuFLn4hyZoPicWczGusrOgOSXyX/AoxvAShTO15jXPT/wt5dvRSkHqbPRI JxjjgC3FKJBrtDgD4p04wO29KHz0YTrNFHLJ9Dna5V0JH/27FlkLOrBqQIJMt2gXK+ Tu7BBXfI+YLfCq7ujPzMHTnWqyMBPIYs3fuiDFreSVnjPY8IH0iOM9/eO5NJn8gqTa 4zSbSNmo771KTlOGtqx8HndvgzOc35lX2+DTMVCR61I0sfqLBFAktDlJOnJ6To1xSI sxx1srD/Gcmyw== Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2546E65662 for ; Thu, 26 May 2022 10:27:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="IGI+yBIn"; dkim-atps=neutral Received: by mail-pg1-x52d.google.com with SMTP id t28so780981pga.6 for ; Thu, 26 May 2022 01:27:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0gtp0n5AGG9baCyOKosgUNXFvcX7/SkZljrUyM+yYjg=; b=IGI+yBInkUfjQftuBOC1GLxGW2jX3mikCdPTPyuKo7+eeIZRXeeSQR9Mi9wdSB7AU2 5jkiMlJ1TTwOUGOoZ3hTvRgJ8UqOOnlqg3AziTHrl6HBh3hLZrZWUWJXrWMAw3PUShZa Ru4BuJO59tNOcnGHS+1pmfq13YMEMa89tae6Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0gtp0n5AGG9baCyOKosgUNXFvcX7/SkZljrUyM+yYjg=; b=STyVM+gaHbHtKsif1DPSosaRagLoQxcxMNA8rR7c+OQEpiK377mEwM+hJghrLSGcvf gs23Focg/GWOJXGi/TYgUxW0McG9Tp99yF2tGyqw9CoJiekMnne96GHzKRSCCVvbMxYB yiDDKI8CnSEf2CUWRDlmCbQHHTvKIiwf4F57Z0CZqRplrdz68QSNQOUrrx/SZ0SrOe67 KMb4OYiLgCOryp2Fm4K2N9VcUX9NrR5qQ6NgxG8QcR46Cno6gIZywIggeddrOXGrlXMk /l07odaISp1kw8WASPvqDq+/Lx6ophD853srIeGDGaeft1GrJ5PAkZor57v/nCD4wR10 0uPg== X-Gm-Message-State: AOAM533k8/gLNAMLUoGDHcQvkGH/HDa76ghmhjVQX1JgeBGT6wUo3h8q fCKFKv5c761KwmScSqgps3Hb/I5td1jXnhIX X-Google-Smtp-Source: ABdhPJz6uO0tV8dW0qi2TLbeds36VVV321Ys1hLhb7CgzgqECdnQ5eH886mrDqw6Yhpg2n4Ncd/d2A== X-Received: by 2002:a63:8a4b:0:b0:3fa:e447:ecc with SMTP id y72-20020a638a4b000000b003fae4470eccmr5463329pgd.399.1653553670942; Thu, 26 May 2022 01:27:50 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:50 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:32 +0000 Message-Id: <20220526082737.1081262-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/9] ipu3: Add StillCapture stream and imgu1 param buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch adds the StillCapture stream and imgu1 param buffers. The following patches will enable imgu1 to handle StillCapture stream specifically, when the imgu0 is needed to handle video/preview streams. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/frames.cpp | 23 +++++++++++++++++++++-- src/libcamera/pipeline/ipu3/frames.h | 8 +++++++- src/libcamera/pipeline/ipu3/ipu3.cpp | 20 +++++++++++++++++++- 3 files changed, 47 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/frames.cpp b/src/libcamera/pipeline/ipu3/frames.cpp index a4c3477c..f9b705a5 100644 --- a/src/libcamera/pipeline/ipu3/frames.cpp +++ b/src/libcamera/pipeline/ipu3/frames.cpp @@ -23,7 +23,8 @@ IPU3Frames::IPU3Frames() } void IPU3Frames::init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers) + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers) { for (const std::unique_ptr &buffer : paramBuffers) availableParamBuffers_.push(buffer.get()); @@ -31,6 +32,9 @@ void IPU3Frames::init(const std::vector> ¶mBuff for (const std::unique_ptr &buffer : statBuffers) availableStatBuffers_.push(buffer.get()); + for (const std::unique_ptr &buffer : captureParamBuffers) + availableCaptureParamBuffers_.push(buffer.get()); + frameInfo_.clear(); } @@ -38,6 +42,7 @@ void IPU3Frames::clear() { availableParamBuffers_ = {}; availableStatBuffers_ = {}; + availableCaptureParamBuffers_ = {}; } IPU3Frames::Info *IPU3Frames::create(Request *request) @@ -54,14 +59,22 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) return nullptr; } + if (availableCaptureParamBuffers_.empty()) { + LOG(IPU3, Debug) << "Capture parameters buffer underrun"; + return nullptr; + } + FrameBuffer *paramBuffer = availableParamBuffers_.front(); FrameBuffer *statBuffer = availableStatBuffers_.front(); + FrameBuffer *captureParamBuffer = availableCaptureParamBuffers_.front(); paramBuffer->_d()->setRequest(request); statBuffer->_d()->setRequest(request); + captureParamBuffer->_d()->setRequest(request); availableParamBuffers_.pop(); availableStatBuffers_.pop(); + availableCaptureParamBuffers_.pop(); /* \todo Remove the dynamic allocation of Info */ std::unique_ptr info = std::make_unique(); @@ -71,7 +84,9 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) info->rawBuffer = nullptr; info->paramBuffer = paramBuffer; info->statBuffer = statBuffer; + info->captureParamBuffer = captureParamBuffer; info->paramDequeued = false; + info->captureParamDequeued = false; info->metadataProcessed = false; frameInfo_[id] = std::move(info); @@ -84,6 +99,7 @@ void IPU3Frames::remove(IPU3Frames::Info *info) /* Return params and stat buffer for reuse. */ availableParamBuffers_.push(info->paramBuffer); availableStatBuffers_.push(info->statBuffer); + availableCaptureParamBuffers_.push(info->captureParamBuffer); /* Delete the extended frame information. */ frameInfo_.erase(info->id); @@ -102,6 +118,9 @@ bool IPU3Frames::tryComplete(IPU3Frames::Info *info) if (!info->paramDequeued) return false; + if (!info->captureParamDequeued) + return false; + remove(info); bufferAvailable.emit(); @@ -131,7 +150,7 @@ IPU3Frames::Info *IPU3Frames::find(FrameBuffer *buffer) return info; if (info->rawBuffer == buffer || info->paramBuffer == buffer || - info->statBuffer == buffer) + info->statBuffer == buffer || info->captureParamBuffer == buffer) return info; } diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h index 6e3cb915..8fcb8a14 100644 --- a/src/libcamera/pipeline/ipu3/frames.h +++ b/src/libcamera/pipeline/ipu3/frames.h @@ -36,16 +36,20 @@ public: FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *captureParamBuffer; + ControlList effectiveSensorControls; bool paramDequeued; + bool captureParamDequeued; bool metadataProcessed; }; IPU3Frames(); void init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers); + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers); void clear(); Info *create(Request *request); @@ -61,6 +65,8 @@ private: std::queue availableParamBuffers_; std::queue availableStatBuffers_; + std::queue availableCaptureParamBuffers_; + std::map> frameInfo_; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e219f704..a201c5ca 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -70,6 +70,7 @@ public: Stream outStream_; Stream vfStream_; Stream rawStream_; + Stream outCaptureStream_; Rectangle cropRegion_; bool supportsFlips_; @@ -696,6 +697,8 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); + else if (stream == &data->outCaptureStream_) + return imgu1_.output_->exportBuffers(count, buffers); return -EINVAL; } @@ -718,11 +721,17 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->outStream_.configuration().bufferCount, data->vfStream_.configuration().bufferCount, data->rawStream_.configuration().bufferCount, + data->outCaptureStream_.configuration().bufferCount, }); ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; + ret = imgu1_.allocateBuffers(bufferCount); + if (ret < 0) { + imgu0_.freeBuffers(); + return ret; + } /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; @@ -737,9 +746,14 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } + for (const std::unique_ptr &buffer : imgu1_.paramBuffers_) { + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + } + data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_, imgu1_.paramBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -760,6 +774,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) ipaBuffers_.clear(); imgu0_.freeBuffers(); + imgu1_.freeBuffers(); return 0; } @@ -1143,6 +1158,7 @@ int PipelineHandlerIPU3::registerCameras() &data->outStream_, &data->vfStream_, &data->rawStream_, + &data->outCaptureStream_, }; CIO2Device *cio2 = &data->cio2_; @@ -1318,6 +1334,8 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) imgu0_->param_->queueBuffer(info->paramBuffer); imgu0_->stat_->queueBuffer(info->statBuffer); imgu0_->input_->queueBuffer(info->rawBuffer); + + info->captureParamDequeued = true; } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Thu May 26 08:27:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16054 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A561BD160 for ; Thu, 26 May 2022 08:27:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF59765674; Thu, 26 May 2022 10:27:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553677; bh=9kwnulspX2JdSQLq+n+ENLikAD05iVOpEntcGl8F45M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=dH4Pwc5BqiMN2b0pIzyZ2VB87m/fQSidaTVpMZ8iviLYuqB5J/VVZNr/Q1mAKFSTz W0JIR80tueP6YXeYBzZHzlaB+FP/uVz+Cbw8kQIBFRwlsnsBPBuHg9buoSVxPV92sH GuBnF6P83xSP+ubgRkr5DijYS5V1mWlQVgZVMtOefJxHC4ipfeDpN2DElhVEi8x4aG 5638n17F/lqkbQE1TtZbZIRMvKIlPdGeNFT9t8PwLo6mDxn9ZAilk/kxM0tAALhnsc 1jmxlOINbdT9749qRDHarNunq1s1RQ/THcL48hMiZCAil0aLaJDVgphTdQ59H38mMm juubHneu6kofw== Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D40265662 for ; Thu, 26 May 2022 10:27:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="YOHv6UmX"; dkim-atps=neutral Received: by mail-pj1-x102b.google.com with SMTP id nn3-20020a17090b38c300b001e0e091cf03so512361pjb.1 for ; Thu, 26 May 2022 01:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tQ4Bk1UrTuHyfOQZJWawRUAEwBR8cQQ227PIgGGKTiw=; b=YOHv6UmXZoebsEk+Wlf+2cN9ugSFnYsLpRNy9Pv1nvDokjDuu72dPM/LJUbYprSfBl mcUtYWYKiXzmrlZhUXQoi2UBGq/+7W0lx2+y046H1nFcjfUFJVGmnHKIijc7Zzaie8bB kEvBTVnY7yXzy47ofFf83V8+SOZfWWeNpuHqM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tQ4Bk1UrTuHyfOQZJWawRUAEwBR8cQQ227PIgGGKTiw=; b=17V82rnk6fuRO2Gjb0cCiIEw91xnU40LV9v77l12oTvc781WwqqVExrFuCE/3s33Ra Cb3tn3cl48Wb/Z2Z26PNACw8ojzl5qKo4EK7D5ksQ0w7HQSf76X1Ocp5HTV8+qZff2et A4C+wW+O082ekjt8QOmClxHdb9dlH/QCDu+z9nixo9mElyE33OiVL9aaJ9cHKo/clbcs brfToKrcKTBq4dR6W0q1mIvyCSM6EVfGfFRJnFFJK87g7B2Xq5cy+CN2i4rZaQ9bUvgV MkqHh3wuNW43eNFgkQodUK6aGcUiLvNqI2zCfSIF1FrUjdwHr+TZwh6GTmO+4IkaKs8I vsaQ== X-Gm-Message-State: AOAM5321EUN51EZdCvGr5gRL8l1TNgk+vrJJdP5KTvHaNSW1nNLYuGW6 4DQ/nKbFbZNwDNvkXuoqqtY+crPJRgc3A6Yo X-Google-Smtp-Source: ABdhPJzpuBPVUwRfD2XKeTMi9Azj9wc40pxC+tPBLdG8WzdqL87cyh3NrD+uis5uadVtGcwL3ZCosA== X-Received: by 2002:a17:902:8693:b0:161:e28f:f85f with SMTP id g19-20020a170902869300b00161e28ff85fmr34083178plo.17.1653553672682; Thu, 26 May 2022 01:27:52 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:52 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:33 +0000 Message-Id: <20220526082737.1081262-6-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/9] ipu3: Update IPAIPU3Interface::fillParamsBuffer with captureBufferId X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch updates the ipa interface |IPAIPU3Interface::fillParamsBuffer| with additional |captureBufferId| to fill the param buffer for the StillCapture stream as well. Signed-off-by: Harvey Yang --- include/libcamera/ipa/ipu3.mojom | 2 +- src/ipa/ipu3/ipu3.cpp | 21 ++++++++++++++++++--- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- 3 files changed, 21 insertions(+), 5 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d1b1c6b8..d94c344e 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -31,7 +31,7 @@ interface IPAIPU3Interface { unmapBuffers(array ids); [async] queueRequest(uint32 frame, libcamera.ControlList controls); - [async] fillParamsBuffer(uint32 frame, uint32 bufferId); + [async] fillParamsBuffer(uint32 frame, uint32 bufferId, uint32 captureBufferId); [async] processStatsBuffer(uint32 frame, int64 frameTimestamp, uint32 bufferId, libcamera.ControlList sensorControls); }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dd6cfd79..149a3958 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -146,7 +146,8 @@ public: void unmapBuffers(const std::vector &ids) override; void queueRequest(const uint32_t frame, const ControlList &controls) override; - void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) override; void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -508,12 +509,14 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) /** * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number - * \param[in] bufferId ID of the parameter buffer to fill + * \param[in] bufferId ID of the video parameter buffer to fill + * \param[in] captureBufferId ID of the capture parameter buffer to fill * * Algorithms are expected to fill the IPU3 parameter buffer for the next * frame given their most recent processing of the ImgU statistics. */ -void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) +void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -536,6 +539,18 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + for (auto const &algo : algorithms_) + algo->prepare(context_, params); + + // TODO: use different algorithms to set StillCapture params. + it = buffers_.find(captureBufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find capture param buffer!"; + return; + } + mem = it->second.planes()[0]; + params = reinterpret_cast(mem.data()); + params->use = {}; for (auto const &algo : algorithms_) algo->prepare(context_, params); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a201c5ca..a13fb881 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1424,7 +1424,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie(), + info->captureParamBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) From patchwork Thu May 26 08:27:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16055 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8794ABD161 for ; Thu, 26 May 2022 08:28:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E082A6566B; Thu, 26 May 2022 10:27:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553679; bh=NilUXQlh17c9nyL0n8+RDM5lzFv3afTxeR0DKKO/yUc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Q68z+ztW7HXeB2pwq+7KfMGmY4mUAtXHe1nH3zGWk8fr5SRtlmSvHQqMs7RNRCpIw 2hLCmpt0bYGWGyfHREXvaOpA8j+HSphqYyjGuxHYiU3LJ9TSippYZmfFfza8jeGueF aoo98A1LfU+QDK10iPEAwrxfQ3/r7EbJLy0P4DTSOFbbTX6Q267XOCTtXIqbebPoLC LH23jTQmosJWeRSWBEF2BaIX+I/AkFK7giolPQVjLUTYEhryiXX2y544r/X1IHq7Mr TxvzfXhe/dygHWmAglLf17o2fsYD+xD56iGQdHE/cw33od31LPhOjFDhYASDgA9kvl wLUua4334STHw== Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8898265663 for ; Thu, 26 May 2022 10:27:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="N/ovMBc9"; dkim-atps=neutral Received: by mail-pg1-x534.google.com with SMTP id v15so767675pgk.11 for ; Thu, 26 May 2022 01:27:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5E3jNNUwiKdnag9tyxtSoX+l4avZ7kjXyEWrGbZbUFM=; b=N/ovMBc9SLdlw+1RTsBOFATNQtl0quGlnMfhsMKpGSQKqxetcN5qHD1WgbxBC/zaxk 6M32Di47UkCKxGFd1yPhC7jRf7t2LQ+cJCP5ozM9PwWQd7hfIQYFmP/9je+PPdj1Mq9R LpIUc7dEbbt+8BbSoe5pnwg45fmLsxMMtj/I0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5E3jNNUwiKdnag9tyxtSoX+l4avZ7kjXyEWrGbZbUFM=; b=SW+6lrNdFa1qxGLioz6jx4F+Aojv76OoywGEDC3Ek/MOYj70KlA52gTaXjmkXtZAH1 kJf3wQa49nHPgLpIvOuOHAUCX7mKB+SrIpyzr9TS4dKDuP/3EtHcvbmZM64DZjmRmlb6 XlYe8uIyKoYg8WIU2WziKGOiZ9p0xwzxryaEgF2Sbz2bwx0MlI7B5+wO3YWNok+sx8Kx VvPoC9jr6mM1M4LNV2mNfbU6aEgpKDgWlOy84+o/WJyemZYCRomYzRuDUu2tt8LCo5vs bAahwfHfkxdLyPHU2bcEwQTTMdgTCWZ0GGTa4Sse7vVJTREp2xE1Kl6Q80qRwgqvpkrW XNFg== X-Gm-Message-State: AOAM532MIea4LIxYUaUWrmPhMMnvn8gOxQ/ng5HNpaW6R4GlonscfyWS jtmJ7J0qX7rlAhF4JuvIUxvBuHMoy4BhwqM7 X-Google-Smtp-Source: ABdhPJwU4l1HPj99xS2LFdz2oUZgMr9D2gLoEtBtAqF9P5avpwU1l3xrTDQx/d9AqTLpMkL9Ag1Qzw== X-Received: by 2002:a63:dd4c:0:b0:3fa:a80e:8ba0 with SMTP id g12-20020a63dd4c000000b003faa80e8ba0mr11039466pgj.167.1653553674382; Thu, 26 May 2022 01:27:54 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:53 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:34 +0000 Message-Id: <20220526082737.1081262-7-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/9] ipu3: Configure imgu1 when necessary X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch configure and setup imgu1 when still capture and other streams are enabled. If only still capture is enabled, imgu0 is still being used with IPU3PipeModeVideo (i.e. VideoSnapshot) to ensure 3A is working. Still capture stream will be enabled in the following patches. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 173 +++++++++++++++++++++++---- 1 file changed, 149 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a13fb881..ec9d14d1 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -55,9 +55,11 @@ public: int loadIPA(); + void tryReturnBuffer(FrameBuffer *buffer); void imguOutputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); + void captureParamBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); void queuePendingRequests(); void cancelPendingRequests(); @@ -93,6 +95,8 @@ private: void paramsBufferReady(unsigned int id); void setSensorControls(unsigned int id, const ControlList &sensorControls, const ControlList &lensControls); + + std::map bufferReturnCounters; }; class IPU3CameraConfiguration : public CameraConfiguration @@ -106,7 +110,8 @@ public: Status validate() override; const StreamConfiguration &cio2Format() const { return cio2Configuration_; } - const ImgUDevice::PipeConfig imguConfig() const { return pipeConfig_; } + const ImgUDevice::PipeConfig imguConfig0() const { return pipeConfig0_; } + const ImgUDevice::PipeConfig imguConfig1() const { return pipeConfig1_; } /* Cache the combinedTransform_ that will be applied to the sensor */ Transform combinedTransform_; @@ -120,7 +125,8 @@ private: const IPU3CameraData *data_; StreamConfiguration cio2Configuration_; - ImgUDevice::PipeConfig pipeConfig_; + ImgUDevice::PipeConfig pipeConfig0_; + ImgUDevice::PipeConfig pipeConfig1_; }; class PipelineHandlerIPU3 : public PipelineHandler @@ -168,6 +174,8 @@ private: MediaDevice *cio2MediaDev_; MediaDevice *imguMediaDev_; + bool useImgu1_ = false; + Camera *inUseCamera_ = nullptr; std::vector ipaBuffers_; @@ -408,8 +416,8 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); - if (pipeConfig_.isNull()) { + pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); + if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; return Invalid; @@ -519,8 +527,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) IPU3CameraData *data = cameraData(camera); Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; + Stream *outCaptureStream = &data->outCaptureStream_; CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); + useImgu1_ = !imguConfig1.isNull(); + int ret; /* @@ -565,6 +578,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.enableLinks(true); + if (ret) + return ret; + } + /* * Pass the requested stream size to the CIO2 unit and get back the * adjusted format to be propagated to the ImgU output devices. @@ -607,14 +626,20 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream has been requested: return here to skip the ImgU configuration * part. */ - ImgUDevice::PipeConfig imguConfig = config->imguConfig(); - if (imguConfig.isNull()) + ImgUDevice::PipeConfig imguConfig0 = config->imguConfig0(); + if (imguConfig0.isNull()) return 0; - ret = imgu0_.configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig0, &cio2Format); if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.configure(imguConfig1, &cio2Format); + if (ret) + return ret; + } + /* Apply the format to the configured streams output devices. */ StreamConfiguration *mainCfg = nullptr; StreamConfiguration *vfCfg = nullptr; @@ -633,6 +658,15 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; + } else if (stream == outCaptureStream) { + ASSERT(useImgu1_); + ret = imgu1_.configureOutput(cfg, &outputFormat); + if (ret) + return ret; + + ret = imgu1_.configureViewfinder(cfg, &outputFormat); + if (ret) + return ret; } } @@ -648,22 +682,34 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu0_.imgu_->controls()); + ControlList ctrls0(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. - * - * \todo Figure out what the 'Still Capture' mode is meant for, and use - * it accordingly. */ - ctrls.set(V4L2_CID_IPU3_PIPE_MODE, - static_cast(IPU3PipeModeVideo)); - ret = imgu0_.imgu_->setControls(&ctrls); + ctrls0.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeVideo)); + ret = imgu0_.imgu_->setControls(&ctrls0); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; } + if (useImgu1_) { + ControlList ctrls1(imgu1_.imgu_->controls()); + /* + * \todo Figure out what the 'Still Capture' mode is meant for, + * and use it accordingly. + */ + ctrls1.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeStillCapture)); + ret = imgu1_.imgu_->setControls(&ctrls1); + if (ret) { + LOG(IPU3, Error) << "Unable to set pipe_mode control"; + return ret; + } + } + ipa::ipu3::IPAConfigInfo configInfo; configInfo.sensorControls = data->cio2_.sensor()->controls(); @@ -672,8 +718,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.lensControls = lens->controls(); configInfo.sensorInfo = sensorInfo; - configInfo.bdsOutputSize = config->imguConfig().bds; - configInfo.iif = config->imguConfig().iif; + configInfo.bdsOutputSize = config->imguConfig0().bds; + configInfo.iif = config->imguConfig0().iif; ret = data->ipa_->configure(configInfo, &data->ipaControls_); if (ret) { @@ -793,8 +839,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis CIO2Device *cio2 = &data->cio2_; int ret; - imgu0_.input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); + imgu0_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); imgu0_.output_->bufferReady.connect(data, &IPU3CameraData::imguOutputBufferReady); imgu0_.viewfinder_->bufferReady.connect(data, @@ -803,6 +849,16 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis &IPU3CameraData::paramBufferReady); imgu0_.stat_->bufferReady.connect(data, &IPU3CameraData::statBufferReady); + + if (useImgu1_) { + imgu1_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); + imgu1_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu1_.param_->bufferReady.connect(data, + &IPU3CameraData::captureParamBufferReady); + } + /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -834,10 +890,17 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; + if (useImgu1_) { + ret = imgu1_.start(); + if (ret) + goto error; + } + return 0; error: imgu0_.stop(); + imgu1_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); @@ -848,6 +911,11 @@ error: imgu0_.viewfinder_->bufferReady.disconnect(); imgu0_.param_->bufferReady.disconnect(); imgu0_.stat_->bufferReady.disconnect(); + imgu1_.input_->bufferReady.disconnect(); + imgu1_.output_->bufferReady.disconnect(); + imgu1_.param_->bufferReady.disconnect(); + + useImgu1_ = false; inUseCamera_ = nullptr; return ret; @@ -863,6 +931,7 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); ret |= imgu0_.stop(); + ret |= imgu1_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); @@ -874,7 +943,11 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->imgu0_->viewfinder_->bufferReady.disconnect(); data->imgu0_->param_->bufferReady.disconnect(); data->imgu0_->stat_->bufferReady.disconnect(); + data->imgu1_->input_->bufferReady.disconnect(); + data->imgu1_->output_->bufferReady.disconnect(); + data->imgu1_->param_->bufferReady.disconnect(); + useImgu1_ = false; inUseCamera_ = nullptr; } @@ -1320,22 +1393,45 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + bool hasYuv = false; + bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ for (auto it : info->request->buffers()) { const Stream *stream = it.first; FrameBuffer *outbuffer = it.second; - if (stream == &outStream_) + if (stream == &outStream_) { + hasYuv = true; imgu0_->output_->queueBuffer(outbuffer); - else if (stream == &vfStream_) + } else if (stream == &vfStream_) { + hasYuv = true; imgu0_->viewfinder_->queueBuffer(outbuffer); + } else if (stream == &outCaptureStream_) { + hasCapture = true; + + imgu1_->output_->queueBuffer(outbuffer); + } } - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + if (hasYuv) { + bufferReturnCounters[info->rawBuffer] += 1; - info->captureParamDequeued = true; + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } else { + info->paramDequeued = true; + info->metadataProcessed = true; + } + + if (hasCapture) { + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } else { + info->captureParamDequeued = true; + } } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) @@ -1356,6 +1452,15 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) * Buffer Ready slots */ +void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) +{ + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + cio2_.tryReturnBuffer(buffer); +} + /** * \brief Handle buffers completion at the ImgU output * \param[in] buffer The completed buffer @@ -1448,6 +1553,26 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) pipe()->completeRequest(request); } +void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) +{ + IPU3Frames::Info *info = frameInfos_.find(buffer); + if (!info) + return; + + info->captureParamDequeued = true; + + /* + * tryComplete() will delete info if it completes the IPU3Frame. + * In that event, we must have obtained the Request before hand. + * + * \todo Improve the FrameInfo API to avoid this type of issue + */ + Request *request = info->request; + + if (frameInfos_.tryComplete(info)) + pipe()->completeRequest(request); +} + void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { IPU3Frames::Info *info = frameInfos_.find(buffer); From patchwork Thu May 26 08:27:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16056 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D527CC326E for ; Thu, 26 May 2022 08:28:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1202B65664; Thu, 26 May 2022 10:28:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553681; bh=5DWQPKQJNZ3aN5Ll1FqAzZ3Y6qxHFg+oJJ2OYbVcQP0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Sgt1ccCdXCk14a+sinZjG4i8hbEnVi0wQWzyk4YaI8IlMu6c3WVSlImzDT57/nkbp y133ocxmaiXB6nSZcV9plMHUkPcK1vIVg7naWDAXDoc1W+3EnUj54Lo6vWBr8ncdVY 3Ey31f1SU3eM6REqVChYZergJaYkaSM4W4wNKL86tZVRPTGHz/JUNyM5DzL0VZaYem B1kifyI1wqEDI3i0HHMYjINzA5cgzdsKxVzXqF5AjwiDMfuT9YujmT7jcs/izBCAZX Irh85R0jtBead3BACuOve0K1EgfujFkhYykg9YBYMWQ7bqyw4kwEHTOh3/R7WxWZRe fIpj79FeBmYAw== Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5EAD165673 for ; Thu, 26 May 2022 10:27:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="J4wbGulQ"; dkim-atps=neutral Received: by mail-pf1-x433.google.com with SMTP id c65so566293pfb.1 for ; Thu, 26 May 2022 01:27:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=br19rRUuI9uyuNQeeSthuD63ktHSskA8R1maEa7PYJ4=; b=J4wbGulQGvxghZx3kz5mv5KjT3I3MJC8Xhbz8SxnQOiZQzWBpV8EG/939RmowJ9h41 m0gv8RxS77oiH3ubuhHsl+cGMW6ablVS1ASvEd71x/FazmF8chhU1TGvM8bnquPpAys7 v5HFm0O9A5XAS17jQ47fGdH7ztlRB5VXcWTSU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=br19rRUuI9uyuNQeeSthuD63ktHSskA8R1maEa7PYJ4=; b=rfwnUWvk2uCKjAux9aaoYwLw6RNceygEpN7fV+TeGeGibsHbcCwUXLOgZ4eVZxi5zo u3odiHr3G5+LOLKwmWeO2huP5ooiYeLEOVfvn+8oZPyu9kT2PngYMLKOG/1ZyabDpmOE 2KRpQg6hkvYfnClxODvbkrDQ6SmbHaOgyhO7LFlwkh24L3kjo/t/ikNSyFXYIh1ILXj+ 3xeMo0sPaPIThc5QdGQ8+Ofb690jcyrOgSBNwrbbAW1usdF4qdDjCttxnT7UD/mEKFI/ FqEK7VXrxuLvCqfOwXLvwL9DblweSHQJ3RUkHqGBje2IBRAXynpn0LzphkHuBUVRhlMA 1NLw== X-Gm-Message-State: AOAM530fNXlNawPr4Ny3dH6VOOkEp++skCW+56quyPFuGu4+Z7eeHfD9 FF+1uAMr0RFLhpeKL1tU1HNJehW+v5kruOhH X-Google-Smtp-Source: ABdhPJx1LRZX4rvMeyUwyS0su+m0XDZF3zFOVFnTbJyQToijONMaRxO7j270A1Sjm68YYWLN0kcykg== X-Received: by 2002:a63:864a:0:b0:3fa:dca8:ef5e with SMTP id x71-20020a63864a000000b003fadca8ef5emr6279809pgd.38.1653553676574; Thu, 26 May 2022 01:27:56 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:55 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:35 +0000 Message-Id: <20220526082737.1081262-8-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 7/9] ipu3: Assign |outCaptureStream| to StillCapture configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When StillCapture and other non-raw configurations are requested, assigns |outCaptureStream| instead of |outStream| to the StillCapture configuration. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 50 +++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index ec9d14d1..e26c2736 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -254,8 +254,10 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * \todo Clarify the IF and BDS margins requirements. */ unsigned int rawCount = 0; - unsigned int yuvCount = 0; + unsigned int videoCount = 0; + unsigned int stillCount = 0; Size maxYuvSize; + Size maxVideoSize; Size rawSize; for (const StreamConfiguration &cfg : config_) { @@ -264,16 +266,29 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { rawCount++; rawSize.expandTo(cfg.size); + } else if (cfg.streamRole == StreamRole::StillCapture) { + if (stillCount != 0) + return Invalid; + + stillCount++; + maxYuvSize.expandTo(cfg.size); } else { - yuvCount++; + videoCount++; maxYuvSize.expandTo(cfg.size); + maxVideoSize.expandTo(cfg.size); } } - if (rawCount > 1 || yuvCount > 2) { + if (videoCount == 0 && stillCount == 1) { + stillCount = 0; + videoCount = 1; + maxVideoSize.expandTo(maxYuvSize); + } + + if (rawCount > 1 || videoCount > 2) { LOG(IPU3, Debug) << "Camera configuration not supported"; return Invalid; - } else if (rawCount && !yuvCount) { + } else if (rawCount && !stillCount && !videoCount) { /* * Disallow raw-only camera configuration. Currently, ImgU does * not get configured for raw-only streams and has early return @@ -316,6 +331,9 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() ImgUDevice::Pipe pipe{}; pipe.input = cio2Configuration_.size; + ImgUDevice::Pipe pipe1{}; + pipe1.input = cio2Configuration_.size; + /* * Adjust the configurations if needed and assign streams while * iterating them. @@ -380,18 +398,34 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() cfg->stride = info.stride(cfg->size.width, 0, 1); cfg->frameSize = info.frameSize(cfg->size, 1); + if (stillCount == 1 && cfg->streamRole == StreamRole::StillCapture) { + LOG(IPU3, Debug) << "Assigned " + << cfg->toString() + << " to the imgu1 main output"; + cfg->setStream(const_cast(&data_->outCaptureStream_)); + + pipe1.main = cfg->size; + pipe1.viewfinder = pipe1.main; + + pipeConfig1_ = data_->imgu1_->calculatePipeConfig(&pipe1); + if (pipeConfig1_.isNull()) { + LOG(IPU3, Error) << "Failed to calculate pipe configuration: " + << "unsupported resolutions."; + return Invalid; + } /* * Use the main output stream in case only one stream is * requested or if the current configuration is the one * with the maximum YUV output size. */ - if (mainOutputAvailable && - (originalCfg.size == maxYuvSize || yuvCount == 1)) { + } else if (mainOutputAvailable && + (originalCfg.size == maxVideoSize || + videoCount == 1)) { cfg->setStream(const_cast(&data_->outStream_)); mainOutputAvailable = false; pipe.main = cfg->size; - if (yuvCount == 1) + if (videoCount == 1) pipe.viewfinder = pipe.main; LOG(IPU3, Debug) << "Assigned " << cfg->toString() @@ -415,7 +449,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() } /* Only compute the ImgU configuration if a YUV stream has been requested. */ - if (yuvCount) { + if (videoCount) { pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " From patchwork Thu May 26 08:27:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16057 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8BA12C326F for ; Thu, 26 May 2022 08:28:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0C2D165673; Thu, 26 May 2022 10:28:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553682; bh=ZeDNURdjdyolK7MV1DwwFCdS+FkO9cKYEFkxNaUib1g=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=gepzAd7gbYe7nxM8t3wJrIY0+nTP/PT9HbOlLdKa7GW2gpIRohaDDDi+wY6/qDsTY VJOWRLYrd51Pdl5Y6+s9cha3V3w+Tj2zmmLXazctyZzf7K7fp123I+D1URVRvr71Rp bsazaL6TELs6gCFPu1xh0N6eZmK8XjAinkefqOz+75EphWSfJa/PwTE53SS9ibSxby SuSgZT9tiI4G1YZbLhGn6l2oSufSFcs2dhwVgAB5iUoO6TyNnrl10DOxSqoHJaUpmo +wg9Ied7sSuIjvtI4jn/VMY7e1JN0IYU52Sp32ErzjyjM97MLp6Fs7K/IlNRt8C1iO OUQbekmxUX9ng== Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3483565673 for ; Thu, 26 May 2022 10:28:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="il34IYLE"; dkim-atps=neutral Received: by mail-pf1-x436.google.com with SMTP id b135so1104963pfb.12 for ; Thu, 26 May 2022 01:28:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J74eD7TTiFhFOR3n/DIz1MYSG15gDmNy5jy9eAcJETo=; b=il34IYLEnrR9QSYyhsII6+Z2PazHNowBRRARnLprDSTpxzUVIIjZ0qrIvxucFflcYA EZtnug2BWQWt5fYBxnPKAwdByMveJ1cITEUw3Nq40DFuSAQZl2qp+DmjwiBjzQQ0eFe8 aOplZD89eup13xX/vkBwiBzvKmVAM1ElOLkhY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J74eD7TTiFhFOR3n/DIz1MYSG15gDmNy5jy9eAcJETo=; b=QRDcRWO4/P7JkshgFnEdF4L39bRp3ZE4510iX6xKt5VeP0bkP4B4vhDhA5TuNmJsWH 5WjmUbL6CtfXI9uyWixIJOIZClO8CTnZgPOpUSi6LJVIPsjdgepw/Q9RpXgMsORcon2t +m8mnGJv/07VwdOTk2Y5GoEKmLyERUSBoTvKFK3zI/+WL9Hvp2Fi+JMnVWPOzdk0u6mB QFOUKYHE43mGtY57sn88Re5YO1XK0meamFisxWd4dF1Hi0XR30XJD4s+A6O/jHjDdbfR 9BDDrDwTbU1CT5pMYat9blSpsTyjGCOPCXHs7g49ud7gFSeztZCvfd4btTPs81psmSyP VlxQ== X-Gm-Message-State: AOAM5306iHl/5ZHHYk6fziCeB9JhZH0CVPONjkoFd0RUdXfdja6qGhrC hyGejP7vTxTR69oclvDEpdI1bBFRazo2KoMh X-Google-Smtp-Source: ABdhPJxNxC+Vndyjv62GkjEwwDfEN+ggYAAKz0XOt45w8L56yxxPOTS2XJvtuseLnPG8Qg3Y7AdObQ== X-Received: by 2002:a63:b25d:0:b0:3f6:5842:2685 with SMTP id t29-20020a63b25d000000b003f658422685mr29106776pgo.363.1653553678243; Thu, 26 May 2022 01:27:58 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:57 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:36 +0000 Message-Id: <20220526082737.1081262-9-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 8/9] Request StillCapture stream as default in Android adapter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the pipeline handler accepts, the Android adapter requests to handle the StillCapture stream separately, instead of being mapped to an existing video/preview stream. Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 214 ++++++++++++++++++---------------- src/android/camera_device.h | 21 ++++ 2 files changed, 132 insertions(+), 103 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..b8403cc2 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -36,23 +36,6 @@ LOG_DECLARE_CATEGORY(HAL) namespace { -/* - * \struct Camera3StreamConfig - * \brief Data to store StreamConfiguration associated with camera3_stream(s) - * \var streams List of the pairs of a stream requested by Android HAL client - * and CameraStream::Type associated with the stream - * \var config StreamConfiguration for streams - */ -struct Camera3StreamConfig { - struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; - }; - - std::vector streams; - StreamConfiguration config; -}; - /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. @@ -536,16 +519,6 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return -EINVAL; #endif - /* - * Generate an empty configuration, and construct a StreamConfiguration - * for each camera3_stream to add to it. - */ - std::unique_ptr config = camera_->generateConfiguration(); - if (!config) { - LOG(HAL, Error) << "Failed to generate camera configuration"; - return -EINVAL; - } - /* * Clear and remove any existing configuration from previous calls, and * ensure the required entries are available without further @@ -614,93 +587,79 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } + std::unique_ptr config; + /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { - CameraStream::Type type; - int index = -1; - - /* Search for a compatible stream in the non-JPEG ones. */ - for (size_t i = 0; i < streamConfigs.size(); ++i) { - Camera3StreamConfig &streamConfig = streamConfigs[i]; - const auto &cfg = streamConfig.config; - - /* - * \todo The PixelFormat must also be compatible with - * the encoder. - */ - if (cfg.size.width != jpegStream->width || - cfg.size.height != jpegStream->height) - continue; - - LOG(HAL, Info) - << "Android JPEG stream mapped to libcamera stream " << i; - - type = CameraStream::Type::Mapped; - index = i; - - /* - * The source stream will be read by software to - * produce the JPEG stream. - */ - camera3_stream_t *stream = streamConfig.streams[0].stream; - stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; - break; - } + /* Check if the JPEG stream can be produced by software. */ + jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; /* - * Without a compatible match for JPEG encoding we must - * introduce a new stream to satisfy the request requirements. + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. */ - if (index < 0) { - /* - * \todo The pixelFormat should be a 'best-fit' choice - * and may require a validation cycle. This is not yet - * handled, and should be considered as part of any - * stream configuration reworks. - */ - Camera3StreamConfig streamConfig; - streamConfig.config.size.width = jpegStream->width; - streamConfig.config.size.height = jpegStream->height; - streamConfig.config.pixelFormat = formats::NV12; - streamConfigs.push_back(std::move(streamConfig)); - - LOG(HAL, Info) << "Adding " << streamConfig.config.toString() - << " for MJPEG support"; - - type = CameraStream::Type::Internal; - index = streamConfigs.size() - 1; - } - - /* The JPEG stream will be produced by software. */ - jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; - - streamConfigs[index].streams.push_back({ jpegStream, type }); - } + Camera3StreamConfig sConfig; + sConfig.config.streamRole = libcamera::StreamRole::StillCapture; + sConfig.config.size.width = jpegStream->width; + sConfig.config.size.height = jpegStream->height; + sConfig.config.pixelFormat = formats::NV12; + + std::vector streamConfigsCopy(streamConfigs); + streamConfigsCopy.push_back(std::move(sConfig)); + + LOG(HAL, Info) << "Adding " << sConfig.config.toString() + << " for MJPEG support"; + + streamConfigsCopy.back().streams.push_back({ jpegStream, CameraStream::Type::Internal }); + + config = tryValidation(streamConfigsCopy, jpegStream); + if (config) { + streamConfigs = streamConfigsCopy; + } else { + bool foundCompatibleStream = false; + /* Search for a compatible stream in the non-JPEG ones. */ + for (size_t i = 0; i < streamConfigs.size(); ++i) { + Camera3StreamConfig &streamConfig = streamConfigs[i]; + const auto &cfg = streamConfig.config; + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size.width != jpegStream->width || + cfg.size.height != jpegStream->height) + continue; + + LOG(HAL, Info) + << "Android JPEG stream mapped to libcamera stream " << i; + foundCompatibleStream = true; + + /* + * The source stream will be read by software to + * produce the JPEG stream. + */ + camera3_stream_t *stream = streamConfig.streams[0].stream; + stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + streamConfigs[i].streams.push_back({ jpegStream, CameraStream::Type::Mapped }); + break; + } - sortCamera3StreamConfigs(streamConfigs, jpegStream); - for (const auto &streamConfig : streamConfigs) { - config->addConfiguration(streamConfig.config); + if (!foundCompatibleStream) { + LOG(HAL, Error) + << "No compatibleStream found for jpeg"; - for (auto &stream : streamConfig.streams) { - streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); - stream.stream->priv = static_cast(&streams_.back()); + return -EINVAL; + } } } - switch (config->validate()) { - case CameraConfiguration::Valid: - break; - case CameraConfiguration::Adjusted: - LOG(HAL, Info) << "Camera configuration adjusted"; - - for (const StreamConfiguration &cfg : *config) - LOG(HAL, Info) << " - " << cfg.toString(); + if (!config) { + config = tryValidation(streamConfigs, jpegStream); + if (!config) + return -EINVAL; - return -EINVAL; - case CameraConfiguration::Invalid: - LOG(HAL, Info) << "Camera configuration invalid"; - return -EINVAL; } /* @@ -1527,3 +1486,52 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons return resultMetadata; } + +std::unique_ptr CameraDevice::tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream) +{ + /* + * Generate an empty configuration, and construct a StreamConfiguration + * for each camera3_stream to add to it. + */ + std::unique_ptr config = camera_->generateConfiguration(); + if (!config) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + return nullptr; + } + + sortCamera3StreamConfigs(streamConfigs, jpegStream); + for (const auto &streamConfig : streamConfigs) { + config->addConfiguration(streamConfig.config); + + for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, + stream.stream, config->size() - 1); + stream.stream->priv = static_cast(&streams_.back()); + } + } + + switch (config->validate()) { + case CameraConfiguration::Valid: + break; + case CameraConfiguration::Adjusted: + LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config) + LOG(HAL, Info) << " - " << cfg.toString(); + + config.reset(); + break; + case CameraConfiguration::Invalid: + LOG(HAL, Info) << "Camera configuration invalid"; + + config.reset(); + break; + } + + if (!config) + streams_.clear(); + + return config; +} diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..8540e3f5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -34,6 +34,23 @@ class Camera3RequestDescriptor; struct CameraConfigData; +/* + * \struct Camera3StreamConfig + * \brief Data to store StreamConfiguration associated with camera3_stream(s) + * \var streams List of the pairs of a stream requested by Android HAL client + * and CameraStream::Type associated with the stream + * \var config StreamConfiguration for streams + */ +struct Camera3StreamConfig { + struct Camera3Stream { + camera3_stream_t *stream; + CameraStream::Type type; + }; + + std::vector streams; + libcamera::StreamConfiguration config; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -101,6 +118,10 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + std::unique_ptr tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream); + unsigned int id_; camera3_device_t camera3Device_; From patchwork Thu May 26 08:27:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16058 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 68968C3256 for ; Thu, 26 May 2022 08:28:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 242126566B; Thu, 26 May 2022 10:28:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553684; bh=SolrwT390owr9IE5xukmzOUdJCgi/vpHgCaNN/WYO90=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=cD5av0mpw7sWKRW9vfWGMfElia/JWwpOFUDXM129gO23Sj0/5ftHHnIs14jt/4cZP ORTOpMo/u9N/MXS8Gfwrj9aPYsG2GKUNR7KWsoqaMzmVZHej7YXW5XTEa/NiC8mX+z x0AjfE3CZKupFkbLbTzVsYmlZKTAgx54HL9PQ75ryRuF4HehxjfOl1fHwJVwpZ6r1w N1Ii3HrqUSsF4NMVZJfrKLUGbV/6sTyStTW+/9OJqFNT42c7HYE2ns8cBbrC+FwxwZ IyTCXQcFhUUlyuTtkABe3BAzNzTJd450Swg8j0mrR9wO60SWxFnlvEwMa8gpEyWAcf gu1GgbAarJsjQ== Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 411D96567C for ; Thu, 26 May 2022 10:28:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="W3ksR6WY"; dkim-atps=neutral Received: by mail-pl1-x636.google.com with SMTP id m1so889832plx.3 for ; Thu, 26 May 2022 01:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=; b=W3ksR6WY533IuOODdI7RQMeNI4LGN1fK+mzQyAR2mYz8LzPwEY2EqeILqaaaGVd/g+ T4uDYWj/289JvlxA08OM+kcIJqs0+fYTrNUJrQoAqhLrJ2Th+yiVPMY7DtVBLl/PzoTt dF+1t4WrunU/kEezc9H5Qu5tOJYVAlJD2/G0c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=; b=2nM+wP/svnTERQwTKVbwkBW1X3GwAfNsw/5uPsrgsPiBhqv2ABnZu9SUJ/ATWMSM6o /jHSdqNyDEPODykph1FboMPD8LmwTF6N/DFlWypNMvoH7ICVqCJ7l1gx3kxBh5wmh4tP CgNoMeQLcPtSd1kVdZZnl5JQwI6sWHWGrhH+Ip8duQpz9yr0JLAnAneZUsFR3WYvs4Bi rPZLcLaVXftDjT+vHEvGiWRMIhrx+V56hElxRRHrNGqTwniQiB7rHI/mgCIj0BpVOQTr oWpHpezpdqxerO7XxZpl7a6qzTI8nFqocI+UQJsIHwA8SYU11UqvWzNM2JRe/Us+8WyK s2sA== X-Gm-Message-State: AOAM5304/DkH1DEzqX9CEwFGP9niwEk3hsEpmQRwO42UHD0hxKQBnivC ZSFRxqTgL/EDIhW2f0a54l3VC2UI+JTOw2JN X-Google-Smtp-Source: ABdhPJxrPfwOLoECVWro4eCU5SgiM1xxZSjphvu97nT/5nDLJsVnQBItdrUkdODWehug98BR5qpm8w== X-Received: by 2002:a17:90b:164e:b0:1df:99f2:be51 with SMTP id il14-20020a17090b164e00b001df99f2be51mr1396238pjb.59.1653553679949; Thu, 26 May 2022 01:27:59 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:59 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:37 +0000 Message-Id: <20220526082737.1081262-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL, handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e26c2736..8689cf8b 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastRequestSequence_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + data->firstRequest_ = true; + data->lastRequestSequence_ = -1; + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); useImgu1_ = !imguConfig1.isNull(); @@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastRequestSequence_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;