From patchwork Thu Aug 8 10:23:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 20831 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D924C323E for ; Thu, 8 Aug 2024 10:23:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4256A633B5; Thu, 8 Aug 2024 12:23:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Ym353Vd/"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D1AF661955 for ; Thu, 8 Aug 2024 12:23:46 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-3687ea0521cso584447f8f.1 for ; Thu, 08 Aug 2024 03:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1723112626; x=1723717426; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Y930BLZrgr8J1JsGdBBm/oclsAO8xMtxvXVk67d/ojg=; b=Ym353Vd/2qR6oxanpyZfgNv5UZu4ZUyeyaAGcg4vTeBUZ0Kxnj9RSLs7d1EgyCnnzj +wewtu4v9HjgRhgDHQqewFyErdjz328AeFOoXv9OIIZGoKJVXJCFLXMgrEtUF76SqSEu OgbD+o0vWipz+JsnXkWJ7JxpyYrwRUZbfD5m2j8UZK6poNEEcAy6cSX0sT7SbcfhkYYO NiLN5y6NX31nFNCOF1Wrq2hmVfQLteCBFdBfDUch91GGFwg2CO/OhB4VFix1+dypTuMs R6+T5GgViuJR6ScMpWGHhZ/1wzR/szsoym7CsiU4GAw1En3O44vElPNuzQhPcRGv15YN Q0tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723112626; x=1723717426; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Y930BLZrgr8J1JsGdBBm/oclsAO8xMtxvXVk67d/ojg=; b=GfJJK+OR1snmTLrCPPdWnOmR7kXHJspoCgtjJ6S3Prx4O38mmbXkhO8U1dA/kKcHET 6kbvFZ8yKqafWTYrbeDzXV0GxxV6D+tiddsTXMCRhWgnLMeyj+6HXArfWHkHzA8mf5cy IUDlm4q7uIQLbpx1MqnJjhwE5ikKKqBLMgAAc9td122imZ3JALCxJ9rHuEiEZ28VuGyj NsKg4sm5DexyLfLz/1KVjyrNxiFnVneZ8OoHgqrP4wKeLUTjRz2waBq9YDxESYvdzItR vMWP3DR/EWQe8zZEEImwM7Enq7ewl77A38K2xd0SD/g4NEyWBWArwusp0nvOSh3pxDvo 3Pcg== X-Gm-Message-State: AOJu0YwjuYedlFo7tCjWhc+1OwOidok1q5yz3FTut/PPGQFZeTifJVlq CnRsdvK9itPKgtyaFDdODdisIt1Nte8WJqyU5VsbhlbsF74x4bJi/A6ANIEwVhAA2LcgLstP3/h Z X-Google-Smtp-Source: AGHT+IHLjz+vaafRSZIZUmn4T7FB3UD9WqmnC0SS41mV5HH/d+B5r2Y0HBMBq/ROsrIBE/IRyAWlEA== X-Received: by 2002:a5d:48c6:0:b0:368:4226:407b with SMTP id ffacd0b85a97d-36d27592025mr1241115f8f.61.1723112625873; Thu, 08 Aug 2024 03:23:45 -0700 (PDT) Received: from naush-laptop.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36d272290c2sm1404234f8f.92.2024.08.08.03.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Aug 2024 03:23:45 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 0/7] Raspberry Pi: Add controls::rpi::ScalerCrops Date: Thu, 8 Aug 2024 11:23:39 +0100 Message-Id: <20240808102346.13065-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi, This patch series adds preliminary support for setting individual crop rectangles for each ISP output stream. The current ScalerCrop control takes a single rectangle and applies it to all output streams. In this series, we define a new rpi::ScalerCrops vendor control that takes a span of rectangles, one for each output stream. The behaviour of the existing ScalerCrop control remains unchanged. One example of the use of this feature is when generating low resolution images for AI inferencing. Typically, vision neural network models are trained on square dimension image with 1:1 aspect ratio (e.g. 640x640 or 320x320). If an application requests, say, a 1920x1080 (16:9) output stream + 640x640 (1:1) inference stream, then the latter images end up squished and distorted. This affects the performance of the neural network. By allowing separate crops on each stream, the inference stream can preserve the pixel aspect ratio for any requested resolution. Note that VC4 based platforms cannot apply different crops to each output, whereas the PiSP/Pi5 platform can. However, all the infrastructure for this change lives in the core pipeline handler shared by both platforms. Thanks, Naush Naushir Patuck (7): controls: rpi: Add a vendor rpi::ScalerCrops control pipeline: rpi: Remove CameraData::scalerCrop_ pipeline: rpi: Pass crop rectangle as a parameter to platformSetIspCrop() pipeline: rpi: Introduce CameraData::CropParams pipeline: rpi: Track which ISP output is configured for a stream pipeline: rpi: Pass ISP output index into platformSetIspCrop() pipeline: rpi: Handler controls::rpi::ScalerCrops src/ipa/rpi/common/ipa_base.cpp | 2 + src/libcamera/control_ids_rpi.yaml | 15 ++++ .../pipeline/rpi/common/pipeline_base.cpp | 84 +++++++++++++------ .../pipeline/rpi/common/pipeline_base.h | 26 +++++- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 18 ++-- 5 files changed, 112 insertions(+), 33 deletions(-)