From patchwork Wed Aug 27 09:07:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 24237 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2A927BD87C for ; Wed, 27 Aug 2025 09:07:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 160E3692F3; Wed, 27 Aug 2025 11:07:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="nCPb1g2D"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31232613C0 for ; Wed, 27 Aug 2025 11:07:44 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-45b618e067eso4399805e9.1 for ; Wed, 27 Aug 2025 02:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1756285663; x=1756890463; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=DBBpe5+K6tnxVPuQLBB6ru6GrDYTmJQWihCc8f2uYUo=; b=nCPb1g2Dc3Poo4CWCO+LK0kwmm71wn756nWASArqGGYcsDsJyaxUNAq74vC3g++/dW 0M8RAtNBuR7CW5Qmf3F/radPuSb2+Ys8whftoyhVyB9WOdWAWf4j+Jj9J0XYXw40bnKe TLogB04ePEYSpQzruhLb92rSXtSJ1koEc5Sli27/hc/hGO2FVHV83i/XkwmPaigxvNLS uwfWk4KOHN2SAxPAQV53xKcgAVHMClCGR5gTaqkbmLFEOyZ87/MYTXQwbX8j6BVt80GQ pkwTZ7BsxdMew40VlG4edpT4SrUovkhSx0zVIwE7xn3bQk8GQJ4k4xxI4lfNRt16Mz80 djUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756285663; x=1756890463; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DBBpe5+K6tnxVPuQLBB6ru6GrDYTmJQWihCc8f2uYUo=; b=BI75whDdRSUVQce7odbdfXgI+O1sJZANehiWvNf95fPJuPDKwDumDcNa9yYltMfK0K UN1dYqCxxhlXXHfRPz2Ac2A5+5rubsElKdaqha/Dsx8DhZvgjap84YY+MTpXZT1tpV7F OqvqmQa+Cf7PiJ7FM389esA5rRKobzOd0J95I0Rvl8jRV8BUwL7e6UhFI+MfWyhlexxg InZhNSEMTEN5fuiPu72DfRhqZnu3gjNUtCAF0AF/CoeU7W0EcwcvixLY4lbTfNx/KxE3 tH015NpbUFItji7CkTw67ugVt0qPNayrMtCoSf2z4+EWwiFOMcsw7qUWKPYptdOO2ThV PRkw== X-Gm-Message-State: AOJu0YxyvHE41j7E/Ab586QVnhb5lyElCy1j1+zEHMpsQLxN387igzcI fkt4aD87k7ASIm9/Dkaihjpv6EcJ09uVeua/Oe+d0kPlLlk/jQzssbCsOt/EL/lZGzxTclKs7bD +2Bkq X-Gm-Gg: ASbGnctcpHZemsV9I4D1gA0yktYy/cxyb9tei9/rivetEXNpRzClmoImc3CiMIXifJe ScC3Juju+hvFp6N5SbDqGdyTUuClhKcgm7DvZawX+Z0ZzndtQFuMGgUeQwI/tlYfO+7+bqhKnu6 wzQHmQzBiecLRhqS5pXez+vXK2qzo0JfLxE00T5woh5bfoeYzkZ91LeLtSVPoddCn5Lqz1AcA81 jp1kIEduKF+KaZ9DCI4FIngLwEzfNXpmtEZE5t5N9RDXnOyS7pMVilKj8fwWYcG77EIdcxnSjQl Me5f+rhcFA2Q7jnUob6vpAw3eQKtCZjHYqnkUcxm6R+IrA3bVhLSlhkL4bOqHbX8emhBR3QD8TH GBF1GXz7pwUuXELx/Zsi1MNagz9UCwEl0c9Mg6daeppHRLBgr2H0dYUpKloc1+Zv1BZnEq79yxz EtypcbpETdHz1PS7XxdNwvLMIKlbY0J1OLitVPizYnQD2OUTAJZw== X-Google-Smtp-Source: AGHT+IHBOLnKXpC60fElg1azyjodzkOJBNXMvnrQQHj4aVH9DAAkRa5RrSGF9dPr/UKT1sy8LtBrLg== X-Received: by 2002:a05:600c:19ca:b0:45b:47e1:ef7b with SMTP id 5b1f17b1804b1-45b6870e3dfmr47682175e9.17.1756285663087; Wed, 27 Aug 2025 02:07:43 -0700 (PDT) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b6f30fe02sm21498675e9.18.2025.08.27.02.07.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Aug 2025 02:07:42 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [RFC PATCH 00/12] Bayer Re-Processing Date: Wed, 27 Aug 2025 10:07:27 +0100 Message-Id: <20250827090739.86955-1-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi everyone Well, I'm pleased to let folks know that the hoary subject of Bayer Re-Processing is finally making some progress. How much, I'll let you judge for yourselves, but nonetheless I'm hoping that some progress is better than none at all! What do these patches do? There are two main things going on here. Firstly, there is the ability to make a "memory camera". A "memory camera" is the gateway to Bayer re-processing, because it gives you access to your hardware ISP as a memory-to-memory device. You can queue requests to it much as you do with regular cameras, only in this case, the raw stream is an input to the system and not an output (there's a new "raw input" stream role). Secondly, the Raspberry Pi pipeline handler is updated to cope with memory cameras and these slightly different kinds of requests. The first 5 patches or so are mostly introducing memory cameras, how we create them, and adding some general helper classes (such as the CameraSensorMemory to use in place of the CameraSensorRaw) which make the rest of the code work with fewer interventions. They are not, or are not particularly, concerned with the Raspberry Pi (though I do take some Pi code from somewhere and move it to the BayerFormat class). The remaining patches are then pretty much all for the Pi PH and IPA. Having thought initially that I might re-factor stuff more generally, I've backed away from that and am using the same PH for regular and memory cameras. There are some places where I've gathered up the Front End / Back End stuff and moved them into separate functions, which does limit the collateral damage somewhat. There is support only for the PiSP platform (Pi 5) not the older VC4 platform (all other devices). I can't say it's all lovely, but you know, nor is it the biggest skeleton ever to inhabit my closet! As I said, this code runs and works. If you want to try it: You can find a libcamera branch with these patches here: https://github.com/davidplowman/libcamera/tree/reprocessing You can find a program that uses a memory camera (to convert DNG files to JPEGs) here: https://github.com/davidplowman/libcamera-apps/blob/reprocessing/apps/rpicam_convert.cpp (check out and build that code branch). Obviously I'm always wanting feedback, particularly on the not-so-Pi changes - namely how people would like to present memory cameras within libcamera, and have users interact with them. Thanks! David David Plowman (12): libcamera: Infrastructure to ask for "memory" cameras libcamera: Add BayerFormat::toMbusCode pipeline: rpi: pisp: Use the new BayerFormat::toMbusCode() instead libcamera: Add a direction (input or output) to the stream configuration libcamera: sensor: Add CameraSensorMemory class ipa: rpi: cam_helper: Add CamHelperDefault class libcamera: pipeline: rpi: Allow creation of the first "memory" camera pipeline: rpi: Allow generation of raw input configurations libcamera: Don't wait for input streams to complete ipa: rpi: Support memory cameras pipeline: rpi: Support memory cameras pipline: rpi: Support memory cameras processing requests include/libcamera/camera_manager.h | 4 + include/libcamera/internal/bayer_format.h | 1 + include/libcamera/internal/camera_manager.h | 5 + include/libcamera/internal/camera_sensor.h | 2 + .../libcamera/internal/camera_sensor_memory.h | 109 +++++ include/libcamera/internal/meson.build | 1 + include/libcamera/internal/pipeline_handler.h | 8 + include/libcamera/stream.h | 10 + src/ipa/rpi/cam_helper/cam_helper_default.cpp | 45 ++ src/ipa/rpi/cam_helper/meson.build | 1 + src/ipa/rpi/common/ipa_base.cpp | 135 ++++-- src/ipa/rpi/common/ipa_base.h | 5 + src/libcamera/bayer_format.cpp | 124 +++++ src/libcamera/camera_manager.cpp | 40 ++ .../pipeline/rpi/common/pipeline_base.cpp | 99 +++- .../pipeline/rpi/common/pipeline_base.h | 5 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 459 ++++++++++++------ src/libcamera/request.cpp | 4 +- src/libcamera/sensor/camera_sensor_memory.cpp | 237 +++++++++ src/libcamera/sensor/meson.build | 1 + src/libcamera/stream.cpp | 7 +- 21 files changed, 1101 insertions(+), 201 deletions(-) create mode 100644 include/libcamera/internal/camera_sensor_memory.h create mode 100644 src/ipa/rpi/cam_helper/cam_helper_default.cpp create mode 100644 src/libcamera/sensor/camera_sensor_memory.cpp