From patchwork Tue Jan 30 18:13:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 19470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B08ADC32A9 for ; Tue, 30 Jan 2024 18:13:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7580D62131; Tue, 30 Jan 2024 19:13:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="SLzSTIVy"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAEED6212F for ; Tue, 30 Jan 2024 19:13:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706638397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U0bMaMtknryfhkQqNQUEoHRL3v5MS+VNOg+/vLle8rs=; b=SLzSTIVyguKYt4b2AhXy5vmSjYoXlyJAdpZT+9fNEziXBZVoWm0WGq17R68ckz03eJqyhg ktYMQY4IcKEcuZ39QsbMNRZwXdkd9Nxnk3QGvJ+xH8skTubHtxlZHgzPJWZXNrBOumJekM d074TXf3CG0zdNfKimLY76hl3+tZf/o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-J7japHXTMIKBxBC9mitQ8Q-1; Tue, 30 Jan 2024 13:13:14 -0500 X-MC-Unique: J7japHXTMIKBxBC9mitQ8Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 55E1F101A526; Tue, 30 Jan 2024 18:13:14 +0000 (UTC) Received: from shalem.redhat.com (unknown [10.39.192.94]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C76240C122E; Tue, 30 Jan 2024 18:13:13 +0000 (UTC) From: Hans de Goede To: libcamera-devel@lists.libcamera.org Subject: [RFC 1/2] libcamera: debayer: Move sizes() to DebayerCpu implementation Date: Tue, 30 Jan 2024 19:13:10 +0100 Message-ID: <20240130181311.43745-2-hdegoede@redhat.com> In-Reply-To: <20240130181311.43745-1-hdegoede@redhat.com> References: <20240130181311.43745-1-hdegoede@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Ripard , Pavel Machek , Bryan O'Donoghue Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The amount of padding needed for interpolation at the borders depends on the implementation. Stop assuming that all implementations need a pattern-width x pattern-height border around the output size for padding and move the sizes() method out of the base-class. Signed-off-by: Hans de Goede --- .../libcamera/internal/software_isp/debayer.h | 26 +------------------ .../internal/software_isp/debayer_cpu.h | 1 + src/libcamera/software_isp/debayer_cpu.cpp | 26 +++++++++++++++++++ 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer.h b/include/libcamera/internal/software_isp/debayer.h index 9109bf32..ab1ce994 100644 --- a/include/libcamera/internal/software_isp/debayer.h +++ b/include/libcamera/internal/software_isp/debayer.h @@ -94,31 +94,7 @@ public: * * \return The valid size ranges or an empty range if there are none. */ - SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) - { - Size pattern_size = patternSize(inputFormat); - - if (pattern_size.isNull()) - return {}; - - /* - * For debayer interpolation a border of pattern-height x pattern-width - * is kept around the entire image. Combined with a minimum-size of - * pattern-height x pattern-width this means the input-size needs to be - * at least (3 * pattern-height) x (3 * pattern-width). - */ - if (inputSize.width < (3 * pattern_size.width) || - inputSize.height < (3 * pattern_size.height)) { - LOG(Debayer, Warning) - << "Input format size too small: " << inputSize.toString(); - return {}; - } - - return SizeRange(Size(pattern_size.width, pattern_size.height), - Size((inputSize.width - 2 * pattern_size.width) & ~(pattern_size.width - 1), - (inputSize.height - 2 * pattern_size.height) & ~(pattern_size.height - 1)), - pattern_size.width, pattern_size.height); - } + virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; /** * \brief Signals when the input buffer is ready. diff --git a/include/libcamera/internal/software_isp/debayer_cpu.h b/include/libcamera/internal/software_isp/debayer_cpu.h index 52af117f..bcb634d5 100644 --- a/include/libcamera/internal/software_isp/debayer_cpu.h +++ b/include/libcamera/internal/software_isp/debayer_cpu.h @@ -62,6 +62,7 @@ public: strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); void process(FrameBuffer *input, FrameBuffer *output, DebayerParams params); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); /** * \brief Get the file descriptor for the statistics. diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index b6393925..14fc383e 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -945,4 +945,30 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams inputBufferReady.emit(input); } +SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + Size pattern_size = patternSize(inputFormat); + + if (pattern_size.isNull()) + return {}; + + /* + * For debayer interpolation a border of pattern-height x pattern-width + * is kept around the entire image. Combined with a minimum-size of + * pattern-height x pattern-width this means the input-size needs to be + * at least (3 * pattern-height) x (3 * pattern-width). + */ + if (inputSize.width < (3 * pattern_size.width) || + inputSize.height < (3 * pattern_size.height)) { + LOG(Debayer, Warning) + << "Input format size too small: " << inputSize.toString(); + return {}; + } + + return SizeRange(Size(pattern_size.width, pattern_size.height), + Size((inputSize.width - 2 * pattern_size.width) & ~(pattern_size.width - 1), + (inputSize.height - 2 * pattern_size.height) & ~(pattern_size.height - 1)), + pattern_size.width, pattern_size.height); +} + } /* namespace libcamera */ From patchwork Tue Jan 30 18:13:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 19471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3AFA1C32C2 for ; Tue, 30 Jan 2024 18:13:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B23A462143; Tue, 30 Jan 2024 19:13:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="LooDNX5J"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B31A62139 for ; Tue, 30 Jan 2024 19:13:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706638399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sASw2vwQy2sWQkoeAM+sNL96F0p+dRyP247spWsQM/M=; b=LooDNX5JIzkzDKq36vAIa6P28TQWEyAJW/cCg7BPvG1P6x+8SfkROdIZAsTv4pD/ggrQ5A S9eFyLjq8EadpHVZ1mIF1J7yqETCeISAm56QOYcFVGsIHtryyqUMZv51hRQaqsDjuKUd9y jvmADdl7QvFqwS58Hff8DoPRU/B3yWU= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-4QwRf-TgPpWl47Ssynajgw-1; Tue, 30 Jan 2024 13:13:16 -0500 X-MC-Unique: 4QwRf-TgPpWl47Ssynajgw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 810EB3C2A1C2; Tue, 30 Jan 2024 18:13:15 +0000 (UTC) Received: from shalem.redhat.com (unknown [10.39.192.94]) by smtp.corp.redhat.com (Postfix) with ESMTP id 876FA40C106C; Tue, 30 Jan 2024 18:13:14 +0000 (UTC) From: Hans de Goede To: libcamera-devel@lists.libcamera.org Subject: [RFC 2/2] libcamera: debayer_cpu: Stop requiring 2 lines of padding at the top/bottom Date: Tue, 30 Jan 2024 19:13:11 +0100 Message-ID: <20240130181311.43745-3-hdegoede@redhat.com> In-Reply-To: <20240130181311.43745-1-hdegoede@redhat.com> References: <20240130181311.43745-1-hdegoede@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Ripard , Pavel Machek , Bryan O'Donoghue Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since the standard bayer pattern debayer functions now take an array of 3 src pointers (previous-, current- and next-line) we can easily avoid the need to have 2 lines (2 to keep the pattern unchanged) padding at the top / bottom. Instead when debayering the first line, pass the next-line pointer also as previous-line pointer and pass the prev-line pointer as next-line pointer when debayering the last line. This allows the output to be 4 lines higher, which is esp. useful with the current hi556 kernel driver which has a max-height for the bauer data of 722 lines. So before this change it was not possible to generate a standard 1280x720 output. Signed-off-by: Hans de Goede --- .../internal/software_isp/debayer_cpu.h | 3 +- src/libcamera/software_isp/debayer_cpu.cpp | 76 +++++++++++++++---- 2 files changed, 64 insertions(+), 15 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_cpu.h b/include/libcamera/internal/software_isp/debayer_cpu.h index bcb634d5..061e22f0 100644 --- a/include/libcamera/internal/software_isp/debayer_cpu.h +++ b/include/libcamera/internal/software_isp/debayer_cpu.h @@ -79,8 +79,9 @@ public: unsigned int frameSize() { return outputConfig_.frameSize; } private: - void initLinePointers(const uint8_t *linePointers[], const uint8_t *src); + void setupInputMemcpy(const uint8_t *linePointers[]); void shiftLinePointers(const uint8_t *linePointers[], const uint8_t *src); + void memcpyNextLine(const uint8_t *linePointers[]); void process2(const uint8_t *src, uint8_t *dst); void process4(const uint8_t *src, uint8_t *dst); /* 8-bit raw bayer format */ diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 14fc383e..d9a7e34f 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -754,14 +754,10 @@ DebayerCpu::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size return std::make_tuple(stride, stride * size.height); } -void DebayerCpu::initLinePointers(const uint8_t *linePointers[], const uint8_t *src) +void DebayerCpu::setupInputMemcpy(const uint8_t *linePointers[]) { const int patternHeight = inputConfig_.patternSize.height; - for (int i = 0; i < patternHeight; i++) - linePointers[i + 1] = src + - (-patternHeight / 2 + i) * (int)inputConfig_.stride; - if (!enableInputMemcpy_) return; @@ -788,10 +784,14 @@ void DebayerCpu::shiftLinePointers(const uint8_t *linePointers[], const uint8_t linePointers[patternHeight] = src + (patternHeight / 2) * (int)inputConfig_.stride; +} +void DebayerCpu::memcpyNextLine(const uint8_t *linePointers[]) +{ if (!enableInputMemcpy_) return; + const int patternHeight = inputConfig_.patternSize.height; size_t lineLength = (window_.width + 2 * inputConfig_.patternSize.width) * inputConfig_.bpp / 8; int padding = inputConfig_.patternSize.width * inputConfig_.bpp / 8; @@ -803,22 +803,53 @@ void DebayerCpu::shiftLinePointers(const uint8_t *linePointers[], const uint8_t void DebayerCpu::process2(const uint8_t *src, uint8_t *dst) { - const unsigned int y_end = window_.y + window_.height; + unsigned int y_end = window_.y + window_.height; + /* Holds [0] previous- [1] current- [2] next-line */ const uint8_t *linePointers[3]; /* Adjust src to top left corner of the window */ src += window_.y * inputConfig_.stride + window_.x * inputConfig_.bpp / 8; - initLinePointers(linePointers, src); + /* [x] becomes [x - 1] after initial shiftLinePointers() call */ + if (window_.y) { + linePointers[1] = src - inputConfig_.stride; /* previous-line */ + linePointers[2] = src; + } else { + /* window_.y == 0, use the next line as prev line */ + linePointers[1] = src + inputConfig_.stride; + linePointers[2] = src; + /* Last 2 lines also need special handling */ + y_end -= 2; + } + + setupInputMemcpy(linePointers); for (unsigned int y = window_.y; y < y_end; y += 2) { shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); stats_->processLine0(y, linePointers); (this->*debayer0_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); + (this->*debayer1_)(dst, linePointers); + src += inputConfig_.stride; + dst += outputConfig_.stride; + } + + if (window_.y == 0) { + shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); + stats_->processLine0(y_end, linePointers); + (this->*debayer0_)(dst, linePointers); + src += inputConfig_.stride; + dst += outputConfig_.stride; + + shiftLinePointers(linePointers, src); + /* next line may point outside of src, use prev. */ + linePointers[2] = linePointers[0]; (this->*debayer1_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; @@ -828,32 +859,46 @@ void DebayerCpu::process2(const uint8_t *src, uint8_t *dst) void DebayerCpu::process4(const uint8_t *src, uint8_t *dst) { const unsigned int y_end = window_.y + window_.height; + /* + * This holds pointers to [0] 2-lines up [1] 1-line up [2] current-line + * [3] 1-line down [4] 2-lines down. + */ const uint8_t *linePointers[5]; /* Adjust src to top left corner of the window */ src += window_.y * inputConfig_.stride + window_.x * inputConfig_.bpp / 8; - initLinePointers(linePointers, src); + /* [x] becomes [x - 1] after initial shiftLinePointers() call */ + linePointers[1] = src - 2 * inputConfig_.stride; + linePointers[2] = src - inputConfig_.stride; + linePointers[3] = src; + linePointers[4] = src + inputConfig_.stride; + + setupInputMemcpy(linePointers); for (unsigned int y = window_.y; y < y_end; y += 4) { shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); stats_->processLine0(y, linePointers); (this->*debayer0_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); (this->*debayer1_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); stats_->processLine2(y, linePointers); (this->*debayer2_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; shiftLinePointers(linePointers, src); + memcpyNextLine(linePointers); (this->*debayer3_)(dst, linePointers); src += inputConfig_.stride; dst += outputConfig_.stride; @@ -948,18 +993,21 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize) { Size pattern_size = patternSize(inputFormat); + unsigned int border_height = pattern_size.height; if (pattern_size.isNull()) return {}; + /* No need for top/bottom border with a pattern height of 2 */ + if (pattern_size.height == 2) + border_height = 0; + /* - * For debayer interpolation a border of pattern-height x pattern-width - * is kept around the entire image. Combined with a minimum-size of - * pattern-height x pattern-width this means the input-size needs to be - * at least (3 * pattern-height) x (3 * pattern-width). + * For debayer interpolation a border is kept around the entire image + * and the minimum output size is pattern-height x pattern-width. */ if (inputSize.width < (3 * pattern_size.width) || - inputSize.height < (3 * pattern_size.height)) { + inputSize.height < (2 * border_height + pattern_size.height)) { LOG(Debayer, Warning) << "Input format size too small: " << inputSize.toString(); return {}; @@ -967,7 +1015,7 @@ SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize) return SizeRange(Size(pattern_size.width, pattern_size.height), Size((inputSize.width - 2 * pattern_size.width) & ~(pattern_size.width - 1), - (inputSize.height - 2 * pattern_size.height) & ~(pattern_size.height - 1)), + (inputSize.height - 2 * border_height) & ~(pattern_size.height - 1)), pattern_size.width, pattern_size.height); }