From patchwork Tue Aug 10 12:17:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siyuan Fan X-Patchwork-Id: 13280 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 98C7CBD87D for ; Tue, 10 Aug 2021 12:17:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D225168822; Tue, 10 Aug 2021 14:17:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=foxmail.com header.i=@foxmail.com header.b="gZGZlkGD"; dkim-atps=neutral Received: from out162-62-57-210.mail.qq.com (out162-62-57-210.mail.qq.com [162.62.57.210]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 803BA687F0 for ; Tue, 10 Aug 2021 14:17:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1628597868; bh=mXMXe7CWiDVhtsmqftTQA/LJdcJ3kEt425pMngds3h0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=gZGZlkGDQkPYgDLzYhfqvkMNl+P96xxbUNB5awNS4pvs+w2Ru4pgjo3b+zTLM7WgP SDnAopS8Z/EKJkL5QjBX+az0Frr/L118KDNMWgWQQcC5vbc4+M6cqLmI3WCeZgif7g NWjGlWDkKVF9gJ6B7Auidf8mkdos9CneA43wKLyw= Received: from localhost.localdomain ([123.126.82.8]) by newxmesmtplogicsvrsza5.qq.com (NewEsmtp) with SMTP id 46D0AC53; Tue, 10 Aug 2021 20:17:45 +0800 X-QQ-mid: xmsmtpt1628597866tal6d1nll Message-ID: X-QQ-XMAILINFO: NCwFnk+fWy3TPq0AMWOCu+Z+fVTdGL0Bm7UKDgeUI6fFDNyI6NiRkq112BNmXe Myqe3/pGlhdzkezEu9PM4E28C33O3La4KQlepWm1RREvDuZJ65+BO7NJM4Ypnp2nRFKpuJLeQoVe IMelXcFKaUyp60YiTdH8daaLloNCoN1KmRakAdZrxi1+ANrzX/3s6CeRGPrQihwTRzDu5uEOSd7O TdZwD3MmlEJUHfq8Qyw9wKKhfx5xbNysXaQTW3uKWkYsSQd09YuewKyjVca0tSsxDnv2aQo9I3wJ p7ryo3Yn3BIvCqophbCqmvAP3mcIOM+1ggViyLrobHLoYgWW+/LO8+4Up7R8DZ/Vz8WMT2UU4apD 4uL53ACuLyJL5vW31y0eWD945XqY5uMVTGLrKR9xMbBWH0eyzB1Wgf3CEKbG+3DJJi8h6tp8DLK0 f+V05y9j31Ibq5qWDGskKxmE4XBxSo8YdIl6evKhLwox+kqDtBhrwnQ0QFWDX00w/Mkz8hRUZdbd LRVxGGxL8SfyCYyqtbr5AYQJL2YG+wout55b47RNcd9H3Y03Jf+2wnqxvwheTOOOy2ApvjwEn0RO bucelrycTvQNlulbCCJsN8xSw8+6XCQ9Ct6VvDKObmE0gpX4ZUZgbdUnLMMuclmKgd12kS3WTGkM cR9t79Ftzf4YJWY3ySG0lCIkZoRrCNvjcz5EiQK0XWLenj8BZUKCde6oKAHLcmnKRORK1PRvFQIJ 6b7aETXOnuPNbwYsWShVd9Zd2wd1EXsN6cr3lfQTWsDiMlMpZO7r2JBV1rGrbgACoNOfMfBB3ywM XOLmi7P5TEolaIDPF88kJX2G10KhF+xHml9yrz5f0ZMWy7LuGqQuw/KHxC/PmGYkIBFC8n8yumbp MH7dEen47vYdyyEMr8cUqIEK19L7xzdjclggqaJaF6NwnOBZcsFJg/HepFOzSqX+z3evttRV0Tf+ 7JodrJVI6oRkDpJ+2OtpunezTv39cGuOLDecjh1B+d+veukxIoPAvrOBBaec0jfuIB+uMxops= From: Siyuan Fan To: libcamera-devel@lists.libcamera.org Date: Tue, 10 Aug 2021 13:17:39 +0100 X-OQ-MSGID: <20210810121742.11012-2-siyuan.fan@foxmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210810121742.11012-1-siyuan.fan@foxmail.com> References: <20210810121742.11012-1-siyuan.fan@foxmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 1/4] pipeline: isp: The software ISP-based pipeline handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Fan Siyuan Changes in V2: -fix the raw and rgb data flow based queue model in pipeline handler -move buffer alloc and thread to ISP class -Fill metadata information in dstBuffer Signed-off-by: Fan Siyuan --- src/libcamera/pipeline/isp/isp.cpp | 81 ++++++++++++------------------ 1 file changed, 32 insertions(+), 49 deletions(-) diff --git a/src/libcamera/pipeline/isp/isp.cpp b/src/libcamera/pipeline/isp/isp.cpp index e5fbd536..c6b7808c 100644 --- a/src/libcamera/pipeline/isp/isp.cpp +++ b/src/libcamera/pipeline/isp/isp.cpp @@ -5,9 +5,10 @@ * isp.cpp - The software ISP-based pipeline handler */ -#include "isp_processing.h" +#include "../../swisp/isp.h" #include +#include #include #include #include @@ -17,7 +18,6 @@ #include #include -#include "libcamera/base/thread.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -38,7 +38,6 @@ public: ~ISPCameraData() { delete video_; - delete rawBuffers; } int init(); @@ -46,16 +45,16 @@ public: void ISPCompleted(FrameBuffer *buffer); Stream stream_; - ISP isp_; - Thread threadISP_; - int width; - int height; + CPU_ISP isp_; + int width_; + int height_; + + std::vector> rawBuffers_; + std::vector rawQueueBuffers_; + std::vector rgbQueueBuffers_; MediaDevice *media_; V4L2VideoDevice *video_; - std::vector> *tempCopyExportBuffers; - std::vector> *rawBuffers; - std::map bufferPair; }; @@ -158,8 +157,8 @@ int PipelineHandlerISP::configure(Camera *camera, CameraConfiguration *config) format.fourcc = v4l2Format; format.size = cfg.size; - data->width = format.size.width; - data->height = format.size.height; + data->width_ = format.size.width; + data->height_ = format.size.height; int ret = data->video_->setFormat(&format); if (ret) @@ -176,30 +175,8 @@ int PipelineHandlerISP::exportFrameBuffers(Camera *camera, Stream *stream, { unsigned int count = stream->configuration().bufferCount; ISPCameraData *data = cameraData(camera); - - for (unsigned int i = 0; i < count; i++) { - std::string name = "frame-" + std::to_string(i); - const int isp_fd = memfd_create(name.c_str(), 0); - int ret = ftruncate(isp_fd, data->width * data->height * 3); - if (ret < 0) { - LOG(ISP, Error) - << "truncate: " - << strerror(-ret); - return ret; - } - FileDescriptor temp = FileDescriptor(std::move(isp_fd)); - - FrameBuffer::Plane rgbPlane; - rgbPlane.fd = std::move(temp); - rgbPlane.length = data->width * data->height * 3; - - std::vector planes; - planes.push_back(std::move(rgbPlane)); - std::unique_ptr buffer = std::make_unique(std::move(planes)); - buffers->push_back(std::move(buffer)); - } - data->tempCopyExportBuffers = buffers; + count = data->isp_.exportBuffers(buffers, count, data->width_, data->height_); return count; @@ -210,15 +187,16 @@ int PipelineHandlerISP::start(Camera *camera, [[maybe_unused]] const ControlList ISPCameraData *data = cameraData(camera); unsigned int count = data->stream_.configuration().bufferCount; - data->rawBuffers = new std::vector>; - int ret = data->video_->allocateBuffers(count, data->rawBuffers); + int ret = data->video_->allocateBuffers(count, &data->rawBuffers_); if (ret < 0) { LOG(ISP, Error) << strerror(-ret); return ret; } - for (unsigned int i = 0; i < count; i++) - data->bufferPair[data->tempCopyExportBuffers->at(i).get()] = data->rawBuffers->at(i).get(); + for (unsigned int i = 0; i < count; i++) { + data->rawQueueBuffers_.push_back(data->rawBuffers_[i].get()); + } + ret = data->video_->streamOn(); if (ret < 0) { @@ -226,7 +204,7 @@ int PipelineHandlerISP::start(Camera *camera, [[maybe_unused]] const ControlList return ret; } - data->threadISP_.start(); + data->isp_.startThreadISP(); return 0; } @@ -235,8 +213,11 @@ void PipelineHandlerISP::stop(Camera *camera) { ISPCameraData *data = cameraData(camera); - data->threadISP_.exit(); - data->threadISP_.wait(); + if (!(data->rawBuffers_.empty())) { + data->rawBuffers_.clear(); + } + + data->isp_.stopThreadISP(); data->video_->streamOff(); data->video_->releaseBuffers(); @@ -246,15 +227,19 @@ int PipelineHandlerISP::queueRequestDevice(Camera *camera, Request *request) { ISPCameraData *data = cameraData(camera); FrameBuffer *rgbBuffer = request->findBuffer(&data->stream_); + data->rgbQueueBuffers_.push_back(rgbBuffer); if (!rgbBuffer) { LOG(ISP, Error) << "Attempt to queue request with invalid stream"; return -ENOENT; } - int ret = data->video_->queueBuffer(data->bufferPair[rgbBuffer]); + int ret = data->video_->queueBuffer(data->rawQueueBuffers_[0]); if (ret < 0) return ret; + FrameBuffer *temp = data->rawQueueBuffers_[0]; + data->rawQueueBuffers_.erase(data->rawQueueBuffers_.begin()); + data->rawQueueBuffers_.push_back(std::move(temp)); return 0; } @@ -297,10 +282,10 @@ void ISPCameraData::ISPCompleted(FrameBuffer *buffer) void ISPCameraData::bufferReady(FrameBuffer *buffer) { - for (std::map::iterator it = bufferPair.begin(); it != bufferPair.end(); it++) { - if (it->second == buffer) - isp_.invokeMethod(&ISP::processing, ConnectionTypeQueued, buffer, it->first, width, height); - } + LOG(ISP, Debug) << rgbQueueBuffers_[0]->planes()[0].fd.fd(); + isp_.invokeMethod(&CPU_ISP::processing, ConnectionTypeQueued, buffer, rgbQueueBuffers_[0], width_, height_); + rgbQueueBuffers_.erase(rgbQueueBuffers_.begin()); + rgbQueueBuffers_.shrink_to_fit(); } @@ -311,8 +296,6 @@ int ISPCameraData::init() return -ENODEV; video_->bufferReady.connect(this, &ISPCameraData::bufferReady); - - isp_.moveToThread(&threadISP_); isp_.ispCompleted.connect(this, &ISPCameraData::ISPCompleted); return 0; From patchwork Tue Aug 10 12:17:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siyuan Fan X-Patchwork-Id: 13282 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7EE4CC3241 for ; Tue, 10 Aug 2021 12:17:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4EE786888B; Tue, 10 Aug 2021 14:17:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=foxmail.com header.i=@foxmail.com header.b="fF3RuIun"; dkim-atps=neutral Received: from out203-205-221-202.mail.qq.com (out203-205-221-202.mail.qq.com [203.205.221.202]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32E34687F0 for ; Tue, 10 Aug 2021 14:17:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1628597870; bh=MpPBdLHjXJ3P7/mkUaYI/7v+TX5Yp4xG20oBoUfcXag=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fF3RuIunO9WbaEiD94szOF0bfKRXXrtKJE00UHYDsmLqP4403jzDOgzXVr2WleVSD WREsdW2hJ9sLUzPJgbaqYd3Y8IwD4got/JY6mjHp0g6fXYK1VxZBTDlOwiw7uQW0Qu EAl5bC/U5/GBZMP+qrIAVagmy1VQnajiAQssauqE= Received: from localhost.localdomain ([123.126.82.8]) by newxmesmtplogicsvrsza5.qq.com (NewEsmtp) with SMTP id 46D0AC53; Tue, 10 Aug 2021 20:17:45 +0800 X-QQ-mid: xmsmtpt1628597868ts7fxlrfm Message-ID: X-QQ-XMAILINFO: N8r1CgRsoLeWHmeMH9snNJkGbAYLkgsf33FaKFQrjfxE98ni6AGtKCckvU8D31 we5wOK1LGXcKZ37JqIz/OdqiKds1XgkR37r7eBtucS9HRQqzB6mZFx7wtkhau+s7YeXUpA2BylZp C6yDwn7tCNvMwPewwG23uQLrMGETuM5lMwwoyH+7FKDUs5aIX94i77oiEEgfGp5BNSow/x3v/FhU hurZ1JrIgyeQk+haIEXyf4TawxY6orfRT6AynIwoQqUzog9kmzL7itx0d63ZO+MYg6523mXs+kRv zpgYupEAhfV1NWufWIW4ugPeS81ISNwgrFESFHwKpj7KjPkJ23YGmtEvFlsycN5VoO8TyNBZhlK/ EAtYQlH2b7QOoKVoRdMVp7dkqnmMtCVIGZDO3gb9qFkos9cNmUSzmUducu4FhLWSIe7kfku/m7mr tMi27W5KlcsxOYm1WKHAwO1fpyPxWwENMC5WDPAT6O9nTqFgafmm2QV4b7zFIEtK/UbwZQVp4Rh+ mp7nrNNum0GSekRGooWDCppdTil4v+tftWIuNHxnvtcA767b/grafAPUpc/bRT/q+hZ9GNXNZeMs qsQO+Y5BhjHZJrwdjx5m+WzrLxdQIDu5ZKeMwWMvBXtpx8PuR6XGuqFrcaksz7YNAADmmIgubBss xs61i+1FXJtILs7I4Du6cIerDH+NDFEczcyip0ETdPcWhJYhuiXmfANT/JLa/Pc1uAYkeIBwLmmL 2rQogckLks9RBscwsafBg42+MGhRFLPIV7sNDxwB4ytxFewgFKv8UYkxBhZFDOMxaTfn3CUMckDj A7DsHRP13oeoeyzxzetwJ89YoDV9Opz6evxDinvofEYjlfTwWd7Ch61sf7fPCpWRlcmjX0piOeZs BtwF6tOTKyyOE2e8IhkNZYhqt5Mv9dmmZDs1UojliEYO1ZAenrCEt3url9Bo3ogCTqLi7I7ALIaM hajhz/r++caEQyrMNJ0iW1L0J7UhwU8HFTtlAlIOlllKTvjht0REKFqs7wlD5/jPaxz5GtQgQ= From: Siyuan Fan To: libcamera-devel@lists.libcamera.org Date: Tue, 10 Aug 2021 13:17:40 +0100 X-OQ-MSGID: <20210810121742.11012-3-siyuan.fan@foxmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210810121742.11012-1-siyuan.fan@foxmail.com> References: <20210810121742.11012-1-siyuan.fan@foxmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 2/4] libcamera: swisp: Move the softawre ISP process to swisp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Fan Siyuan Changes in V2: -Design the abstract class ISP, CPU_ISP and future GPU_ISP inherit from it. -Add the rgb buf alloc and thread control member function Now the ISP must implement the below process: BLC-->demosaic-->AWB-->tone mapping-->gamma-->Noise reduction Plan to add: lens shading correct-->Color correct-->color space transform Signed-off-by: Fan Siyuan --- .../isp/isp_processing.cpp => swisp/isp.cpp} | 90 ++++++++++++++----- .../isp/isp_processing.h => swisp/isp.h} | 52 ++++++++--- 2 files changed, 111 insertions(+), 31 deletions(-) rename src/libcamera/{pipeline/isp/isp_processing.cpp => swisp/isp.cpp} (87%) rename src/libcamera/{pipeline/isp/isp_processing.h => swisp/isp.h} (54%) diff --git a/src/libcamera/pipeline/isp/isp_processing.cpp b/src/libcamera/swisp/isp.cpp similarity index 87% rename from src/libcamera/pipeline/isp/isp_processing.cpp rename to src/libcamera/swisp/isp.cpp index 94e8081d..6d073040 100644 --- a/src/libcamera/pipeline/isp/isp_processing.cpp +++ b/src/libcamera/swisp/isp.cpp @@ -2,10 +2,10 @@ /* * Copyright (C) 2021, Siyuan Fan * - * isp_processing.cpp - The software ISP class + * isp.cpp - The software ISP class */ -#include "isp_processing.h" +#include "isp.h" #include #include @@ -14,6 +14,7 @@ #include #include +#include #include "libcamera/base/log.h" @@ -21,7 +22,7 @@ namespace libcamera{ LOG_DECLARE_CATEGORY(ISP) -void ISP::autoContrast(uint16_t *data, float lowCut, float highCut, int width, int height) +void CPU_ISP::autoContrast(uint16_t *data, float lowCut, float highCut, int width, int height) { int blue, gr, gb, red; int histBlue[1024] = {0}, histGb[1024] = {0}, histGr[1024] = {0}, histRed[1024] = {0}; @@ -211,7 +212,7 @@ void ISP::autoContrast(uint16_t *data, float lowCut, float highCut, int width, i } } -void ISP::blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) +void CPU_ISP::blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) { int len = width * height; for(int i = 0; i < len; i++) { @@ -224,7 +225,7 @@ void ISP::blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int heig } } -void ISP::readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, +void CPU_ISP::readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) { int index; @@ -245,7 +246,7 @@ void ISP::readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, } } -void ISP::firstPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) +void CPU_ISP::firstPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) { int index; for (int i = 0; i < height; i++) { @@ -295,7 +296,7 @@ void ISP::firstPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) } } -void ISP::twoPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) +void CPU_ISP::twoPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) { int index; for (int i = 0; i < height; i++) { @@ -340,7 +341,7 @@ void ISP::twoPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) } } -void ISP::lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) +void CPU_ISP::lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) { int index; for (int i = 0; i < height; i++) { @@ -390,15 +391,15 @@ void ISP::lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height) } } -void ISP::demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, - int width, int height) +void CPU_ISP::demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, + int width, int height) { firstPixelInsert(data, B, width, height); twoPixelInsert(data, G, width, height); lastPixelInsert(data, R, width, height); } -void ISP::autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) +void CPU_ISP::autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) { float aveB = 0, aveG = 0, aveR = 0; float Kb, Kg, Kr; @@ -428,8 +429,7 @@ void ISP::autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int } } -void ISP::gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, - int width, int height) +void CPU_ISP::gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height) { float nor = 1.0 / 1023.0; float gamma = 1.0 / val; @@ -444,24 +444,24 @@ void ISP::gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, } } -void ISP::compress_10bit_to_8bit (uint16_t *src, uint8_t *dst, int width, int height) +void CPU_ISP::compress_10bit_to_8bit (uint16_t *src, uint8_t *dst, int width, int height) { for (int i = 0; i < width * height; i++) { dst[i] = src[i] >> 2 & 0xff; } } -float ISP::distance(int x, int y, int i, int j) +float CPU_ISP::distance(int x, int y, int i, int j) { return float(sqrt(pow(x - i, 2) + pow(y - j, 2))); } -double ISP::gaussian(float x, double sigma) +double CPU_ISP::gaussian(float x, double sigma) { return exp(-(pow(x, 2)) / (2 * pow(sigma, 2))) / (2 * 3.1415926 * pow(sigma, 2)); } -void ISP::bilateralFilter(uint16_t *R, uint16_t *G, uint16_t *B, +void CPU_ISP::bilateralFilter(uint16_t *R, uint16_t *G, uint16_t *B, int diameter, double sigmaI, double sigmaS, int width, int height) { @@ -541,7 +541,12 @@ void ISP::bilateralFilter(uint16_t *R, uint16_t *G, uint16_t *B, } } -void ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, int height) +void CPU_ISP::noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) +{ + bilateralFilter(R, G, B, 5, 24.0, 32.0, width, height); +} + +void CPU_ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, int height) { uint8_t *rgb_buf; uint16_t *rawData; @@ -554,6 +559,7 @@ void ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, const FrameBuffer::Plane &plane = srcBuffer->planes()[0]; rawData = (uint16_t *)mmap(NULL, plane.length, PROT_READ|PROT_WRITE, MAP_SHARED, plane.fd.fd(), 0); + LOG(ISP, Debug) << plane.length << " " << plane.fd.fd(); if (rawData == MAP_FAILED) { LOG(ISP, Error) << "Read raw data failed"; ispCompleted.emit(dstBuffer); @@ -567,7 +573,7 @@ void ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, autoContrast(gData, 0.01, 0.01, width, height); autoContrast(bData, 0.01, 0.01, width, height); gammaCorrect(rData, gData, bData, 2.2, width, height); - //bilateralFilter(rData, gData, bData, 5, 24.0, 32.0); + //bilateralFilter(rData, gData, bData, 5, 24.0, 32.0, width, height); const FrameBuffer::Plane &rgbPlane = dstBuffer->planes()[0]; rgb_buf = (uint8_t *)mmap(NULL, rgbPlane.length, PROT_READ|PROT_WRITE, MAP_SHARED, rgbPlane.fd.fd(), 0); @@ -584,10 +590,54 @@ void ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, compress_10bit_to_8bit(gData, gData_8, width, height); compress_10bit_to_8bit(bData, bData_8, width, height); + dstBuffer->metadata_.status = srcBuffer->metadata().status; + dstBuffer->metadata_.sequence = srcBuffer->metadata().sequence; + dstBuffer->metadata_.timestamp = srcBuffer->metadata().timestamp; + + dstBuffer->metadata_.planes.clear(); + dstBuffer->metadata_.planes.push_back({rgbPlane.length}); + delete[] rgbData; ispCompleted.emit(dstBuffer); } -} /* namespace libcamera */ +int CPU_ISP::exportBuffers(std::vector> *buffers, + unsigned int count, int width, int height) +{ + int bufferByte = width * height * 3; + + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int isp_fd = memfd_create(name.c_str(), 0); + int ret = ftruncate(isp_fd, bufferByte); + if (ret < 0) { + LOG(ISP, Error) << "Failed to resize memfd" << strerror(-ret); + return ret; + } + + FrameBuffer::Plane rgbPlane; + rgbPlane.fd = FileDescriptor(std::move(isp_fd)); + rgbPlane.length = bufferByte; + + std::vector planes{rgbPlane}; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +void CPU_ISP::startThreadISP() +{ + moveToThread(&thread_); + thread_.start(); +} + +void CPU_ISP::stopThreadISP() +{ + thread_.exit(); + thread_.wait(); +} +} /* namespace libcamera */ \ No newline at end of file diff --git a/src/libcamera/pipeline/isp/isp_processing.h b/src/libcamera/swisp/isp.h similarity index 54% rename from src/libcamera/pipeline/isp/isp_processing.h rename to src/libcamera/swisp/isp.h index b80b4218..4099e2c6 100644 --- a/src/libcamera/pipeline/isp/isp_processing.h +++ b/src/libcamera/swisp/isp.h @@ -2,15 +2,16 @@ /* * Copyright (C) 2021, Siyuan Fan * - * isp_processing.h - The software ISP class + * isp.h - The software ISP class */ -#ifndef __LIBCAMERA_PIPELINE_ISP_PROCESSING_H__ -#define __LIBCAMERA_PIPELINE_ISP_PROCESSING_H__ +#ifndef __LIBCAMERA_SWISP_ISP_H__ +#define __LIBCAMERA_SWISP_ISP_H__ #include -#include "libcamera/base/signal.h" #include "libcamera/base/object.h" +#include "libcamera/base/signal.h" +#include "libcamera/base/thread.h" namespace libcamera{ @@ -19,10 +20,29 @@ using std::uint8_t; class ISP : public Object { +public: + ISP() {} + + virtual void blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) = 0; + + virtual void demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, + int width, int height) = 0; + + virtual void autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) = 0; + + virtual void gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height) = 0; + + virtual void noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) = 0; + + virtual ~ISP() {} +}; + +class CPU_ISP : public ISP +{ public: void autoContrast(uint16_t *data, float lowCut, float highCut, int width, int height); - void blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height); + void blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) override; void readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, int width, int height); @@ -34,12 +54,11 @@ public: void lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height); void demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B, - int width, int height); + int width, int height) override; - void autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height); + void autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) override; - void gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, - int width, int height); + void gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height) override; float distance(int x, int y, int i, int j); @@ -49,14 +68,25 @@ public: int diameter, double sigmaI, double sigmaS, int width, int height); + void noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) override; + void compress_10bit_to_8bit(uint16_t *src, uint8_t *dst, int width, int height); void processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, int height); + int exportBuffers(std::vector> *buffers, + unsigned int count, int width, int height); + + void startThreadISP(); + + void stopThreadISP(); + Signal ispCompleted; +private: + Thread thread_; }; -} +} /* namespace libcamera */ -#endif /* __LIBCAMERA_PIPELINE_ISP_PROCESSING_H__ */ +#endif /* __LIBCAMERA_SWISP_ISP_H__ */ \ No newline at end of file From patchwork Tue Aug 10 12:17:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siyuan Fan X-Patchwork-Id: 13283 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0989C3241 for ; Tue, 10 Aug 2021 12:18:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B1D476882A; Tue, 10 Aug 2021 14:18:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=foxmail.com header.i=@foxmail.com header.b="mEBvhadV"; dkim-atps=neutral Received: from out203-205-221-164.mail.qq.com (out203-205-221-164.mail.qq.com [203.205.221.164]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 18FC068887 for ; Tue, 10 Aug 2021 14:17:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1628597872; bh=EIqLODHjrEbAGuJ2JGkHGKc4vTVmI0zsqYyym2mcLZc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=mEBvhadVBRwG8SvphRFPfUc7aLmIRsn5UT1VG6udOfCJEbFETTc+lAD2HHW1+dIKZ soVHML0jx22SV7KJqvlqCTksPTZUiH8FMFWgudmBMoFCejRGZdVHQY3y+GxALuCoqz wAASlEcfCa4mTaWfM3Da+RnfBP8CYSsvpAhha3cU= Received: from localhost.localdomain ([123.126.82.8]) by newxmesmtplogicsvrsza5.qq.com (NewEsmtp) with SMTP id 46D0AC53; Tue, 10 Aug 2021 20:17:45 +0800 X-QQ-mid: xmsmtpt1628597870tmcibvffj Message-ID: X-QQ-XMAILINFO: NqN/wpVFVRYX/pYJZjhb+dUPh4sQOUIUwq688lIf6HQcmrE9mcyYnRVaUWM1Sl WMMLXf/xmufXtEV7eg4CwFBTBGtTJ24kRJXjVWMjuOOIR9F83l0O3/Pkfng4p5D6ThaTnwUBNcnU QlpEHAoTmrB2ejIOPVMQVQsaFVsqaM8/aL6L0oWxnJp85qnbpi4MoGqdXsO2RWBxtlZOjV9Nw2CW C1Cl9c3UtvSjgS6313gwfESozdO0UxAixRUtbTWSZBRt9gpgqEwgu2J1W7jwBtwY/G4nO5NWJhzO onImw7hr0ET3CQz0xg4RJujZfG6g0nxlFZf+NtKylG34o+nRqIoV9W1+gvRvkMLrOP3LRsv+Wm6P hF80a2gW4BiaT+o39a9qnxWky4chJvz+dSlpLoTppbOf9/xzLGKDxkCg5abF2G22N16yVGzGBGkY 9Jyzx501IpYbx6TQwJLwVO/fiEYd41qJFObWBPnjZQcv+acNYdPZiBPCpiKaJ2yJ21NgW+Erb93d q4Vu+uxVUhkkbPu1auvIpj9HcXO9Wp3kWcy6Js2V8PwBCv8m2FJe2Fj1rxg9Cx0dZNQORx5px0tg 9tMylbBI+6b+Evdc572HOChbFiO2GSFMYgddfy3smQzSpHqtj73AI71nDBkUuYacWAPNrp4P86lG Ae+OPtXh7fHm0+XmOqr7CbjaT8lHAQyUAadbBODUcFs2Yuv2QkeOlL+HRDm5UIiYqZV6HqhLDKfW J1nAVZ06dmL7skZXQuLl5dK3FFhrSqeWNcAWq0l1K486Lplt9PjIp/3vSWoqlyooSlZ1T5A1V9nH 38qe5WDSf3fNaPPWDxhqyGb/6dwm7VKeel9DUHfSxTnFCt2GsFrqxA54jqoolv3mpZvPYFUckRQe +dAiAJKEt4aoZIv5nxIF5PTYMLwoR+AjeFQSNXIpby08VcU72MiSUhggBXfrKRLW9Kq1mu4sWZCS TSH8qsk1LhzZ/QIVLbkg== From: Siyuan Fan To: libcamera-devel@lists.libcamera.org Date: Tue, 10 Aug 2021 13:17:41 +0100 X-OQ-MSGID: <20210810121742.11012-4-siyuan.fan@foxmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210810121742.11012-1-siyuan.fan@foxmail.com> References: <20210810121742.11012-1-siyuan.fan@foxmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 3/4] libcamera: framebuffer: Add the friend class CPU_ISP X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Fan Siyuan In order to fill in dstBuffer->metadata(), we need to add the friend class CPU_ISP. Signed-off-by: Fan Siyuan --- include/libcamera/framebuffer.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index 28307890..9cc1b760 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -61,6 +61,7 @@ private: LIBCAMERA_DISABLE_COPY_AND_MOVE(FrameBuffer) friend class V4L2VideoDevice; /* Needed to update metadata_. */ + friend class CPU_ISP; std::vector planes_;