From patchwork Thu Feb 27 10:57:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22884 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 185D3BF415 for ; Thu, 27 Feb 2025 10:57:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4DC5A68756; Thu, 27 Feb 2025 11:57:42 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5329468749 for ; Thu, 27 Feb 2025 11:57:40 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:57:36 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 01/11] include: linux: Add C3 ISP Stats and Params V4L2 format Date: Thu, 27 Feb 2025 18:57:23 +0800 Message-ID: <20250227105733.187611-2-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the new format describing the C3 ISP Statistics and Parameters to videodev2. These come from the v6 of the C3 ISP kernel driver series[1]. Once the kernel driver is merged we can replace these temporary manual additions with updates to the scripts that merge in the kernel headers. [1] https://lore.kernel.org/linux-media/20250227-c3isp-v6-6-f72e19084d0d@amlogic.com/ Signed-off-by: Keke Li --- include/linux/videodev2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 317d063a..a9f73438 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -843,6 +843,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */ #define V4L2_META_FMT_MALI_C55_3A_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ +/* Vendor specific - used for C3 ISP */ +#define V4L2_META_FMT_C3ISP_PARAMS v4l2_fourcc('C', '3', 'P', 'M') /* Amlogic C3 ISP Parameters */ +#define V4L2_META_FMT_C3ISP_STATS v4l2_fourcc('C', '3', 'S', 'T') /* Amlogic C3 ISP Statistics */ + /* * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when * adding new ones! From patchwork Thu Feb 27 10:57:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22885 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4CB40BF415 for ; Thu, 27 Feb 2025 10:57:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8DEBB6875B; Thu, 27 Feb 2025 11:57:46 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EEE366874F for ; Thu, 27 Feb 2025 11:57:40 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:57:37 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 02/11] libcamera: pipeline: Add c3-isp pipeline handler Date: Thu, 27 Feb 2025 18:57:24 +0800 Message-ID: <20250227105733.187611-3-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Amlogic C3 ISP pipeline handler supports 3-channel image output, 1-channel 3A statistical information ouput and 1-channel parameters input. Signed-off-by: Keke Li --- Documentation/Doxyfile-common.in | 1 + include/libcamera/ipa/c3-isp.mojom | 34 + include/libcamera/ipa/meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/c3-isp/c3-isp.cpp | 1313 +++++++++++++++++++++ src/libcamera/pipeline/c3-isp/meson.build | 5 + 6 files changed, 1355 insertions(+) create mode 100644 include/libcamera/ipa/c3-isp.mojom create mode 100644 src/libcamera/pipeline/c3-isp/c3-isp.cpp create mode 100644 src/libcamera/pipeline/c3-isp/meson.build diff --git a/Documentation/Doxyfile-common.in b/Documentation/Doxyfile-common.in index 045c19dd..19f28886 100644 --- a/Documentation/Doxyfile-common.in +++ b/Documentation/Doxyfile-common.in @@ -31,6 +31,7 @@ RECURSIVE = YES EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \ @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \ + @TOP_BUILDDIR@/include/libcamera/ipa/c3-isp_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/mali-c55_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/raspberrypi_*.h \ diff --git a/include/libcamera/ipa/c3-isp.mojom b/include/libcamera/ipa/c3-isp.mojom new file mode 100644 index 00000000..f8556256 --- /dev/null +++ b/include/libcamera/ipa/c3-isp.mojom @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +module ipa.c3isp; + +import "include/libcamera/ipa/core.mojom"; + +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; +}; + +interface IPAC3ISPInterface { + init(libcamera.IPASettings settings, IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers, bool readOnly); + unmapBuffers(array buffers); + + [async] queueRequest(uint32 request, libcamera.ControlList reqControls); + [async] computeParams(uint32 request, uint32 bufferId); + [async] processStats(uint32 request, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPAC3ISPEventInterface { + paramsComputed(uint32 request, uint32 bytesused); + statsProcessed(uint32 request, libcamera.ControlList metadata); + setSensorControls(libcamera.ControlList sensorControls); +}; diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index 3129f119..bc0fa7d3 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -63,6 +63,7 @@ libcamera_ipa_headers += custom_target('core_ipa_serializer_h', # Mapping from pipeline handler name to mojom file pipeline_ipa_mojom_mapping = { + 'c3-isp': 'c3-isp.mojom', 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', diff --git a/meson_options.txt b/meson_options.txt index f19bca91..998a4463 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -46,6 +46,7 @@ option('pipelines', choices : [ 'all', 'auto', + 'c3-isp', 'imx8-isi', 'ipu3', 'mali-c55', diff --git a/src/libcamera/pipeline/c3-isp/c3-isp.cpp b/src/libcamera/pipeline/c3-isp/c3-isp.cpp new file mode 100644 index 00000000..90baedc8 --- /dev/null +++ b/src/libcamera/pipeline/c3-isp/c3-isp.cpp @@ -0,0 +1,1313 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * Pipeline Handler for Amlogic C3 ISP + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace { + +bool isFmtRaw(const libcamera::PixelFormat &pixFmt) +{ + return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + +} /* namespace */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(C3ISP) + +const std::map C3ISPFmtToCode = { + { formats::R8, MEDIA_BUS_FMT_YUV10_1X30 }, + { formats::NV12, MEDIA_BUS_FMT_YUV10_1X30 }, + { formats::NV21, MEDIA_BUS_FMT_YUV10_1X30 }, + { formats::NV16, MEDIA_BUS_FMT_YUV10_1X30 }, + { formats::NV61, MEDIA_BUS_FMT_YUV10_1X30 }, + + /* RAW formats, STILL pipe only. */ + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB16_1X16 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR16_1X16 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG16_1X16 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG16_1X16 }, +}; + +constexpr Size kC3ISPMinSize = { 160, 120 }; +constexpr Size kC3ISPMaxSize = { 2888, 2240 }; + +struct C3ISPFrameInfo { + Request *request; + + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + + bool paramsDone; + bool statsDone; +}; + +class C3ISPCameraData : public Camera::Private +{ +public: + C3ISPCameraData(PipelineHandler *pipe, MediaEntity *entity) + : Camera::Private(pipe), entity_(entity) + { + } + + int init(); + int loadIPA(); + + int pixfmtToMbusCode(const PixelFormat &pixFmt) const; + const PixelFormat &bestRawFmt() const; + + PixelFormat adjustRawFmt(const PixelFormat &pixFmt) const; + Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; + + void updateControls(const ControlInfoMap &ipaControls); + + MediaEntity *entity_; + std::unique_ptr sensor_; + std::unique_ptr csi_; + std::unique_ptr adap_; + Stream viewStream; + Stream stillStream; + Stream videoStream; + + std::unique_ptr ipa_; + std::vector ipaStatBuffers_; + std::vector ipaParamBuffers_; + + std::unique_ptr delayedCtrls_; + +private: + void setSensorControls(const ControlList &sensorControls); +}; + +int C3ISPCameraData::init() +{ + /* Register a CameraSensor */ + sensor_ = CameraSensorFactoryBase::create(entity_); + if (!sensor_) + return -EINVAL; + + const MediaPad *sensorSrc = entity_->getPadByIndex(0); + MediaEntity *csiEntity = sensorSrc->links()[0]->sink()->entity(); + + csi_ = std::make_unique(csiEntity); + if (csi_->open()) { + LOG(C3ISP, Error) << "Failed to open CSI-2 subdevice"; + return -EINVAL; + } + + const MediaPad *csiSrc = csiEntity->getPadByIndex(1); + MediaEntity *adapEntity = csiSrc->links()[0]->sink()->entity(); + + adap_ = std::make_unique(adapEntity); + if (adap_->open()) { + LOG(C3ISP, Error) << "Failed to open adapter subdevice"; + return -EINVAL; + } + + return 0; +} + +void C3ISPCameraData::setSensorControls(const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +/* + * Search the sensor's supported formats for the one with a matching bayer + * order and the greatest bitdepth. + */ +int C3ISPCameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const +{ + auto it = C3ISPFmtToCode.find(pixFmt); + if (it == C3ISPFmtToCode.end()) + return -EINVAL; + + BayerFormat bayerFmt = BayerFormat::fromMbusCode(it->second); + if (!bayerFmt.isValid()) + return -EINVAL; + + unsigned int snsMbusCode = 0; + unsigned int bitDepth = 0; + for (const auto &code : sensor_->mbusCodes()) { + BayerFormat snsBayerFmt = BayerFormat::fromMbusCode(code); + if (!snsBayerFmt.isValid()) + continue; + + if (snsBayerFmt.order != bayerFmt.order) + continue; + + if (snsBayerFmt.bitDepth > bitDepth) { + bitDepth = snsBayerFmt.bitDepth; + snsMbusCode = code; + } + } + + if (!snsMbusCode) + return -EINVAL; + + return snsMbusCode; +} + +/* Find a RAW PixelFormat supported by both the ISP and the sensor. */ +const PixelFormat &C3ISPCameraData::bestRawFmt() const +{ + static const PixelFormat invalidPixFmt = {}; + + for (const auto &code : sensor_->mbusCodes()) { + BayerFormat sensorBayer = BayerFormat::fromMbusCode(code); + + if (!sensorBayer.isValid()) + continue; + + for (const auto &[pixFmt, rawCode] : C3ISPFmtToCode) { + if (!isFmtRaw(pixFmt)) + continue; + + BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); + if (bayer.order == sensorBayer.order) + return pixFmt; + } + } + + LOG(C3ISP, Error) << "Can't get a compatible format"; + + return invalidPixFmt; +} + +/* + * Make sure the provided raw pixel format is supported and adjust it to + * one of the supported ones if it's not. + */ +PixelFormat C3ISPCameraData::adjustRawFmt(const PixelFormat &rawFmt) const +{ + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) + return bestRawFmt(); + + const auto rawSizes = sensor_->sizes(rawCode); + if (rawSizes.empty()) + return bestRawFmt(); + + return rawFmt; +} + +Size C3ISPCameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const +{ + Size rawSize = size.boundedTo(kC3ISPMaxSize); + + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) + return {}; + + const auto rawSizes = sensor_->sizes(rawCode); + + auto sizeIt = std::find(rawSizes.begin(), rawSizes.end(), rawSize); + if (sizeIt != rawSizes.end()) + return rawSize; + + /* Adjust the rawSize to the closest supported size */ + uint16_t distance = std::numeric_limits::max(); + Size bestSize; + for (const Size &sz : rawSizes) { + uint16_t dist = std::abs(static_cast(rawSize.width) - + static_cast(sz.width)) + + std::abs(static_cast(rawSize.height) - + static_cast(sz.height)); + if (dist < distance) { + distance = dist; + bestSize = sz; + } + } + + return bestSize; +} + +void C3ISPCameraData::updateControls(const ControlInfoMap &ipaControls) +{ + ControlInfoMap::Map controls; + + for (auto const &c : ipaControls) + controls.emplace(c.first, c.second); + + controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); +} + +int C3ISPCameraData::loadIPA() +{ + int ret; + + ipa_ = IPAManager::createIPA(pipe(), 1, 1); + if (!ipa_) + return -ENOENT; + + ipa_->setSensorControls.connect(this, &C3ISPCameraData::setSensorControls); + + std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml", + "uncalibrated.yaml"); + + ipa::c3isp::IPAConfigInfo ipaConfig{}; + + ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + if (ret) + return ret; + + ipaConfig.sensorControls = sensor_->controls(); + + ControlInfoMap ipaControls; + ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, &ipaControls); + if (ret) { + LOG(C3ISP, Error) << "Failed to initialize IPA"; + return ret; + } + + updateControls(ipaControls); + + return 0; +} + +class C3ISPCameraConfiguration : public CameraConfiguration +{ +public: + C3ISPCameraConfiguration(C3ISPCameraData *data) + : CameraConfiguration(), data_(data) + { + } + + Status validate() override; + const Transform &combinedTransform() { return combinedTransform_; } + + V4L2SubdeviceFormat sensorFormat_; + +private: + static constexpr unsigned int kMaxStreams = 3; + + const C3ISPCameraData *data_; + Transform combinedTransform_; +}; + +CameraConfiguration::Status C3ISPCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) + return Invalid; + + if (config_.size() > kMaxStreams) { + config_.resize(kMaxStreams); + status = Adjusted; + } + + Orientation requestOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestOrientation) + status = Adjusted; + + bool stillPipeAvailable = true; + StreamConfiguration *rawConfig = nullptr; + for (StreamConfiguration &config : config_) { + if (!isFmtRaw(config.pixelFormat)) + continue; + + if (rawConfig) { + LOG(C3ISP, Error) << "Only support a RAW stream"; + return Invalid; + } + + rawConfig = &config; + } + + /* + * The C3 ISP can not upscale. Limit the configuration to the ISP + * capabilities and the sensor resolution. + */ + Size maxSize = kC3ISPMaxSize.boundedTo(data_->sensor_->resolution()); + if (rawConfig) { + PixelFormat rawFmt = + data_->adjustRawFmt(rawConfig->pixelFormat); + + if (!rawFmt.isValid()) + return Invalid; + + if (rawFmt != rawConfig->pixelFormat) { + LOG(C3ISP, Debug) + << "Adjust RAW format to " << rawFmt; + rawConfig->pixelFormat = rawFmt; + status = Adjusted; + } + + Size rawSize = data_->adjustRawSizes(rawFmt, rawConfig->size); + if (rawSize != rawConfig->size) { + LOG(C3ISP, Debug) + << "Adjust RAW size to " << rawSize; + rawConfig->size = rawSize; + status = Adjusted; + } + + maxSize = rawSize; + + const PixelFormatInfo &info = PixelFormatInfo::info(rawConfig->pixelFormat); + rawConfig->stride = info.stride(rawConfig->size.width, 0, 16); + rawConfig->frameSize = info.frameSize(rawConfig->size, 4); + + rawConfig->setStream(const_cast(&data_->stillStream)); + stillPipeAvailable = false; + } + + /* Adjust processed streams */ + + bool videoPipeAvailable = true; + Size maxConfigSize; + for (StreamConfiguration &config : config_) { + if (isFmtRaw(config.pixelFormat)) + continue; + + const auto it = C3ISPFmtToCode.find(config.pixelFormat); + if (it == C3ISPFmtToCode.end()) { + LOG(C3ISP, Debug) + << "Format adjusted to " << formats::NV12; + config.pixelFormat = formats::NV12; + status = Adjusted; + } + + Size size = std::clamp(config.size, kC3ISPMinSize, maxSize); + if (size != config.size) { + LOG(C3ISP, Debug) + << "Size adjusted to " << size; + config.size = size; + status = Adjusted; + } + + if (maxConfigSize < size) + maxConfigSize = size; + + if (stillPipeAvailable) { + config.setStream(const_cast(&data_->stillStream)); + stillPipeAvailable = false; + } else if (videoPipeAvailable) { + config.setStream(const_cast(&data_->videoStream)); + videoPipeAvailable = false; + } else { + config.setStream(const_cast(&data_->viewStream)); + } + } + + /* Configure the sensor format */ + + /* If there's a RAW config, sensor format follow it */ + if (rawConfig) { + sensorFormat_.code = data_->pixfmtToMbusCode(rawConfig->pixelFormat); + sensorFormat_.size = rawConfig->size; + + LOG(C3ISP, Debug) << "RAW configuration format " << sensorFormat_; + + return status; + } + + /* If there's no RAW config, compture the sensor format */ + PixelFormat rawFormat = data_->bestRawFmt(); + if (!rawFormat.isValid()) + return Invalid; + + sensorFormat_.code = data_->pixfmtToMbusCode(rawFormat); + sensorFormat_.size = data_->adjustRawSizes(rawFormat, maxConfigSize); + + LOG(C3ISP, Debug) << "Sensor format " << sensorFormat_; + + return status; +} + +class PipelineHandlerC3ISP : public PipelineHandler +{ +public: + PipelineHandlerC3ISP(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + Span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int allocateBuffers(Camera *camera); + void freeBuffers(Camera *camera); + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + void imageBufferReady(FrameBuffer *buffer); + void paramsBufferReady(FrameBuffer *buffer); + void statsBufferReady(FrameBuffer *buffer); + void paramsComputed(unsigned int requestId, unsigned int bytesused); + void statsProcessed(unsigned int requestId, const ControlList &metadata); + + bool match(DeviceEnumerator *enumerator) override; + +private: + struct C3ISPPipe { + std::unique_ptr resizer; + std::unique_ptr cap; + Stream *stream; + }; + + enum { + C3ISPVIEW, + C3ISPVIDEO, + C3ISPSTILL, + C3ISPNumPipes, + }; + + C3ISPCameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + C3ISPPipe *pipeFromStream(C3ISPCameraData *data, Stream *stream) + { + if (stream == &data->viewStream) + return &pipes_[C3ISPVIEW]; + else if (stream == &data->stillStream) + return &pipes_[C3ISPSTILL]; + else if (stream == &data->videoStream) + return &pipes_[C3ISPVIDEO]; + else + LOG(C3ISP, Fatal) << "Invalid stream: " << stream; + + return nullptr; + } + + C3ISPPipe *pipeFromStream(C3ISPCameraData *data, const Stream *stream) + { + return pipeFromStream(data, const_cast(stream)); + } + + void resetPipes() + { + for (C3ISPPipe &pipe : pipes_) + pipe.stream = nullptr; + } + + int pipesStart(); + void pipesStop(); + + int configureRawStream(C3ISPCameraData *data, + const StreamConfiguration &config, + V4L2SubdeviceFormat &subdevFormat); + int configureProcessedStream(C3ISPCameraData *data, + const StreamConfiguration &config, + V4L2SubdeviceFormat &subdevFormat); + + bool createCamera(MediaLink *ispLink); + void tryComplete(C3ISPFrameInfo *info); + C3ISPFrameInfo *findFrameInfo(FrameBuffer *buffer); + C3ISPFrameInfo *findFrameInfo(Request *request); + void clearIncompleteRequests(); + + MediaDevice *media_; + std::unique_ptr isp_; + std::unique_ptr params_; + std::unique_ptr stats_; + + std::vector> statsBuffers_; + std::queue availableStatsBuffers_; + + std::vector> paramsBuffers_; + std::queue availableParamsBuffers_; + + std::map frameInfoMap_; + + std::array pipes_; +}; + +PipelineHandlerC3ISP::PipelineHandlerC3ISP(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +std::unique_ptr +PipelineHandlerC3ISP::generateConfiguration(Camera *camera, + Span roles) +{ + C3ISPCameraData *data = cameraData(camera); + std::unique_ptr config = + std::make_unique(data); + bool isRoleRaw = false; + + if (roles.empty()) + return config; + + /* Reserve the C3ISPSTILL pipe for Raw role */ + if (std::find(roles.begin(), roles.end(), StreamRole::Raw) != roles.end()) + isRoleRaw = true; + + for (const StreamRole &role : roles) { + struct C3ISPPipe *pipe; + PixelFormat pixelFormat; + Size size = std::min(Size{ 1920, 1080 }, data->sensor_->resolution()); + + switch (role) { + case StreamRole::StillCapture: + size = data->sensor_->resolution(); + pixelFormat = formats::NV12; + pipe = isRoleRaw ? nullptr : &pipes_[C3ISPSTILL]; + break; + + case StreamRole::VideoRecording: + pixelFormat = formats::NV12; + pipe = &pipes_[C3ISPVIDEO]; + break; + + case StreamRole::Viewfinder: + pixelFormat = formats::NV12; + pipe = &pipes_[C3ISPVIEW]; + break; + + case StreamRole::Raw: + pixelFormat = data->bestRawFmt(); + if (!pixelFormat.isValid()) { + LOG(C3ISP, Error) + << "Failed to get Raw format"; + return nullptr; + } + + size = data->sensor_->resolution(); + pipe = &pipes_[C3ISPSTILL]; + break; + + default: + LOG(C3ISP, Error) << "Invalid stream role: " << role; + return nullptr; + } + + std::map> formats; + for (const auto &c3Format : C3ISPFmtToCode) { + PixelFormat pixFmt = c3Format.first; + bool isRaw = isFmtRaw(pixFmt); + + if (pipe != &pipes_[C3ISPSTILL] && isRaw) + continue; + + if (isRaw) { + int rawCode = data->pixfmtToMbusCode(pixFmt); + if (rawCode < 0) + continue; + + const auto sizes = data->sensor_->sizes(rawCode); + if (sizes.empty()) + continue; + + std::vector sizeRanges; + std::transform(sizes.begin(), sizes.end(), + std::back_inserter(sizeRanges), + [](const Size &s) { + return SizeRange(s); + }); + + formats[pixFmt] = sizeRanges; + } else { + Size maxSize = std::min(kC3ISPMaxSize, + data->sensor_->resolution()); + formats[pixFmt] = { kC3ISPMinSize, maxSize }; + } + } + + StreamFormats streamFormats(formats); + StreamConfiguration cfg(streamFormats); + cfg.pixelFormat = pixelFormat; + cfg.bufferCount = 4; + cfg.size = size; + + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + return nullptr; + + return config; +} + +int PipelineHandlerC3ISP::configureRawStream(C3ISPCameraData *data, + const StreamConfiguration &config, + V4L2SubdeviceFormat &subdevFormat) +{ + Stream *stream = config.stream(); + C3ISPPipe *pipe = pipeFromStream(data, stream); + + if (pipe != &pipes_[C3ISPSTILL]) { + LOG(C3ISP, Error) << "Failed to match the RAW pipe"; + return -EINVAL; + } + + const MediaEntity *resizerEntity = pipe->resizer->entity(); + int ret = resizerEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + if (ret) { + LOG(C3ISP, Error) << "Couldn't enable resizer's sink link"; + return ret; + } + + auto it = C3ISPFmtToCode.find(config.pixelFormat); + if (it == C3ISPFmtToCode.end()) { + LOG(C3ISP, Error) << "Failed to find the RAW pixel format"; + return -EINVAL; + } + + subdevFormat.code = it->second; + ret = isp_->setFormat(5, &subdevFormat); + if (ret) + return ret; + + ret = pipe->resizer->setFormat(0, &subdevFormat); + if (ret) + return ret; + + return 0; +} + +int PipelineHandlerC3ISP::configureProcessedStream(C3ISPCameraData *data, + const StreamConfiguration &config, + V4L2SubdeviceFormat &subdevFormat) +{ + Stream *stream = config.stream(); + C3ISPPipe *pipe = pipeFromStream(data, stream); + + const MediaEntity *resizerEntity = pipe->resizer->entity(); + int ret = resizerEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + if (ret) + return ret; + + auto it = C3ISPFmtToCode.find(config.pixelFormat); + if (it == C3ISPFmtToCode.end()) { + LOG(C3ISP, Error) << "Failed to find the processed pixel format"; + return -EINVAL; + } + + unsigned int pad; + if (pipe == &pipes_[C3ISPVIEW]) + pad = 3; + else if (pipe == &pipes_[C3ISPVIDEO]) + pad = 4; + else if (pipe == &pipes_[C3ISPSTILL]) + pad = 5; + else { + LOG(C3ISP, Error) << "Failed to match the pipe"; + return -EINVAL; + } + + subdevFormat.code = it->second; + ret = isp_->setFormat(pad, &subdevFormat); + if (ret) + return ret; + + ret = pipe->resizer->setFormat(0, &subdevFormat); + if (ret) + return ret; + + /* Compute the scaler-in to scaler-out ratio: first center-crop to align + * the FOV to the desired resolution, then scale to the desired size. + */ + Size scalerIn = subdevFormat.size.boundedToAspectRatio(config.size); + int xCrop = (subdevFormat.size.width - scalerIn.width) / 2; + int yCrop = (subdevFormat.size.height - scalerIn.height) / 2; + + Rectangle cropRect = { xCrop, yCrop, scalerIn }; + ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_CROP, &cropRect); + if (ret) + return ret; + + Rectangle scaleRect = { 0, 0, config.size }; + ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &scaleRect); + if (ret) + return ret; + + subdevFormat.size = scaleRect.size(); + + return pipe->resizer->setFormat(1, &subdevFormat); +} + +int PipelineHandlerC3ISP::configure(Camera *camera, CameraConfiguration *config) +{ + resetPipes(); + + int ret = media_->disableLinks(); + if (ret) + return ret; + + C3ISPCameraData *data = cameraData(camera); + + /* Enable the link between sensor and CSI-2 */ + const MediaEntity *csiEntity = data->csi_->entity(); + ret = csiEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + if (ret) + return ret; + + /* Enable the link between CSI-2 and adapter */ + const MediaEntity *adapEntity = data->adap_->entity(); + ret = adapEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + if (ret) + return ret; + + /* Enable the link between adapter and ISP */ + const MediaEntity *ispEntity = isp_->entity(); + ret = ispEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + if (ret) + return ret; + + C3ISPCameraConfiguration *c3Config = static_cast(config); + V4L2SubdeviceFormat subdevFormat = c3Config->sensorFormat_; + + ret = data->sensor_->setFormat(&subdevFormat, c3Config->combinedTransform()); + if (ret) + return ret; + + ret = data->csi_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + ret = data->adap_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + ret = isp_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + for (const StreamConfiguration &streamConfig : *config) { + Stream *stream = streamConfig.stream(); + C3ISPPipe *pipe = pipeFromStream(data, stream); + + if (isFmtRaw(streamConfig.pixelFormat)) + ret = configureRawStream(data, streamConfig, subdevFormat); + else + ret = configureProcessedStream(data, streamConfig, subdevFormat); + if (ret) { + LOG(C3ISP, Error) << "Failed to configure pipeline"; + return ret; + } + + V4L2DeviceFormat captureFormat; + captureFormat.fourcc = pipe->cap->toV4L2PixelFormat(streamConfig.pixelFormat); + captureFormat.size = streamConfig.size; + + ret = pipe->cap->setFormat(&captureFormat); + if (ret) + return ret; + + pipe->stream = stream; + } + + /* Enable the link between parameter node and ISP */ + ret = ispEntity->getPadByIndex(1)->links()[0]->setEnabled(true); + if (ret) { + LOG(C3ISP, Error) << "Failed to enable param link"; + return ret; + } + + /* Enable the link between ISP and 3A stats node */ + ret = ispEntity->getPadByIndex(2)->links()[0]->setEnabled(true); + if (ret) { + LOG(C3ISP, Error) << "Failed to enable stats link"; + return ret; + } + + /* Inform the IPA of the sensor configuration */ + ipa::c3isp::IPAConfigInfo ipaConfig{}; + + ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + if (ret) + return ret; + + ipaConfig.sensorControls = data->sensor_->controls(); + + ControlInfoMap ipaControls; + ret = data->ipa_->configure(ipaConfig, &ipaControls); + if (ret) { + LOG(C3ISP, Error) << "Failed to configure IPA"; + return ret; + } + + data->updateControls(ipaControls); + + return 0; +} + +int PipelineHandlerC3ISP::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + C3ISPPipe *pipe = pipeFromStream(cameraData(camera), stream); + unsigned int count = stream->configuration().bufferCount; + + return pipe->cap->exportBuffers(count, buffers); +} + +int PipelineHandlerC3ISP::allocateBuffers(Camera *camera) +{ + C3ISPCameraData *data = cameraData(camera); + unsigned int ipaBufferId = 1; + unsigned int bufferCount; + int ret; + + bufferCount = std::max({ data->viewStream.configuration().bufferCount, + data->videoStream.configuration().bufferCount, + data->stillStream.configuration().bufferCount }); + + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : statsBuffers_) { + buffer->setCookie(ipaBufferId++); + data->ipaStatBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); + availableStatsBuffers_.push(buffer.get()); + } + + ret = params_->allocateBuffers(bufferCount, ¶msBuffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : paramsBuffers_) { + buffer->setCookie(ipaBufferId++); + data->ipaParamBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); + availableParamsBuffers_.push(buffer.get()); + } + + data->ipa_->mapBuffers(data->ipaStatBuffers_, true); + data->ipa_->mapBuffers(data->ipaParamBuffers_, false); + + return 0; +} + +void PipelineHandlerC3ISP::freeBuffers(Camera *camera) +{ + C3ISPCameraData *data = cameraData(camera); + + while (!availableStatsBuffers_.empty()) + availableStatsBuffers_.pop(); + while (!availableParamsBuffers_.empty()) + availableParamsBuffers_.pop(); + + statsBuffers_.clear(); + paramsBuffers_.clear(); + + data->ipa_->unmapBuffers(data->ipaStatBuffers_); + data->ipa_->unmapBuffers(data->ipaParamBuffers_); + + data->ipaStatBuffers_.clear(); + data->ipaParamBuffers_.clear(); + + if (stats_->releaseBuffers()) + LOG(C3ISP, Error) << "Failed to release stats buffers"; + + if (params_->releaseBuffers()) + LOG(C3ISP, Error) << "Failed to release params buffers"; +} + +int PipelineHandlerC3ISP::pipesStart() +{ + for (C3ISPPipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + Stream *stream = pipe.stream; + + int ret = pipe.cap->importBuffers(stream->configuration().bufferCount); + if (ret) { + LOG(C3ISP, Error) << "Failed to import buffers"; + return ret; + } + + ret = pipe.cap->streamOn(); + if (ret) { + LOG(C3ISP, Error) << "Failed to start stream"; + return ret; + } + } + + return 0; +} + +void PipelineHandlerC3ISP::pipesStop() +{ + for (C3ISPPipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + pipe.cap->streamOff(); + pipe.cap->releaseBuffers(); + } +} + +int PipelineHandlerC3ISP::start(Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + C3ISPCameraData *data = cameraData(camera); + int ret; + + ret = allocateBuffers(camera); + if (ret) + return ret; + + ret = data->ipa_->start(); + if (ret) { + LOG(C3ISP, Error) << "Failed to start IPA"; + + freeBuffers(camera); + return ret; + } + + ret = pipesStart(); + if (ret) { + LOG(C3ISP, Error) << "Failed to start pipe"; + + data->ipa_->stop(); + freeBuffers(camera); + return ret; + } + + ret = stats_->streamOn(); + if (ret) { + LOG(C3ISP, Error) << "Failed to start stats"; + + pipesStop(); + data->ipa_->stop(); + freeBuffers(camera); + return ret; + } + + ret = params_->streamOn(); + if (ret) { + LOG(C3ISP, Error) << "Failed to start params"; + + stats_->streamOff(); + pipesStop(); + data->ipa_->stop(); + freeBuffers(camera); + return ret; + } + + ret = isp_->setFrameStartEnabled(true); + if (ret) { + LOG(C3ISP, Error) << "Failed to enable frame start"; + + params_->streamOff(); + stats_->streamOff(); + pipesStop(); + data->ipa_->stop(); + freeBuffers(camera); + return ret; + } + + return 0; +} + +void PipelineHandlerC3ISP::stopDevice(Camera *camera) +{ + C3ISPCameraData *data = cameraData(camera); + + isp_->setFrameStartEnabled(false); + + params_->streamOff(); + stats_->streamOff(); + data->ipa_->stop(); + freeBuffers(camera); + + pipesStop(); + clearIncompleteRequests(); +} + +int PipelineHandlerC3ISP::queueRequestDevice(Camera *camera, Request *request) +{ + C3ISPCameraData *data = cameraData(camera); + + if (availableStatsBuffers_.empty()) { + LOG(C3ISP, Error) << "No available stats buffer"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(C3ISP, Error) << "No available params buffer"; + return -ENOENT; + } + + C3ISPFrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + data->ipa_->queueRequest(request->sequence(), request->controls()); + data->ipa_->computeParams(request->sequence(), + frameInfo.paramBuffer->cookie()); + + return 0; +} + +C3ISPFrameInfo *PipelineHandlerC3ISP::findFrameInfo(Request *request) +{ + for (auto &[sequence, info] : frameInfoMap_) { + if (info.request == request) + return &info; + } + + return nullptr; +} + +C3ISPFrameInfo *PipelineHandlerC3ISP::findFrameInfo(FrameBuffer *buffer) +{ + for (auto &[sequence, info] : frameInfoMap_) { + if (info.paramBuffer == buffer || info.statBuffer == buffer) + return &info; + } + + return nullptr; +} + +void PipelineHandlerC3ISP::clearIncompleteRequests() +{ + for (auto &[sequence, info] : frameInfoMap_) { + if (info.request) + cancelRequest(info.request); + } + + frameInfoMap_.clear(); +} + +void PipelineHandlerC3ISP::tryComplete(C3ISPFrameInfo *info) +{ + if (!info->paramsDone) + return; + + if (!info->statsDone) + return; + + Request *request = info->request; + if (request->hasPendingBuffers()) + return; + + if (info->statBuffer) + availableStatsBuffers_.push(info->statBuffer); + + if (info->paramBuffer) + availableParamsBuffers_.push(info->paramBuffer); + + frameInfoMap_.erase(request->sequence()); + + completeRequest(request); +} + +void PipelineHandlerC3ISP::imageBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + C3ISPFrameInfo *frameInfo = findFrameInfo(request); + ASSERT(frameInfo); + + if (completeBuffer(request, buffer)) + tryComplete(frameInfo); +} + +void PipelineHandlerC3ISP::paramsBufferReady(FrameBuffer *buffer) +{ + C3ISPFrameInfo *frameInfo = findFrameInfo(buffer); + ASSERT(frameInfo); + + frameInfo->paramsDone = true; + + tryComplete(frameInfo); +} + +void PipelineHandlerC3ISP::statsBufferReady(FrameBuffer *buffer) +{ + C3ISPFrameInfo *frameInfo = findFrameInfo(buffer); + ASSERT(frameInfo); + + Request *request = frameInfo->request; + C3ISPCameraData *data = cameraData(request->_d()->camera()); + + ControlList sensorControls = data->delayedCtrls_->get(buffer->metadata().sequence); + + data->ipa_->processStats(request->sequence(), buffer->cookie(), + sensorControls); +} + +void PipelineHandlerC3ISP::paramsComputed(unsigned int requestId, + unsigned int bytesused) +{ + C3ISPFrameInfo &frameInfo = frameInfoMap_[requestId]; + Request *request = frameInfo.request; + C3ISPCameraData *data = cameraData(request->_d()->camera()); + + frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; + + params_->queueBuffer(frameInfo.paramBuffer); + stats_->queueBuffer(frameInfo.statBuffer); + + for (auto &[stream, buffer] : request->buffers()) { + C3ISPPipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } +} + +void PipelineHandlerC3ISP::statsProcessed(unsigned int requestId, + const ControlList &metadata) +{ + C3ISPFrameInfo &frameInfo = frameInfoMap_[requestId]; + + frameInfo.statsDone = true; + frameInfo.request->metadata().merge(metadata); + + tryComplete(&frameInfo); +} + +bool PipelineHandlerC3ISP::createCamera(MediaLink *ispLink) +{ + MediaEntity *adap = ispLink->source()->entity(); + const MediaPad *adapSink = adap->getPadByIndex(0); + MediaEntity *csi = adapSink->links()[0]->source()->entity(); + const MediaPad *csiSink = csi->getPadByIndex(0); + + MediaEntity *sensor = csiSink->links()[0]->source()->entity(); + if (sensor->function() != MEDIA_ENT_F_CAM_SENSOR) + return false; + + std::unique_ptr data = + std::make_unique(this, sensor); + if (data->init()) + return false; + + /* Generic values for sensor */ + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + if (data->loadIPA()) + return false; + + data->ipa_->statsProcessed.connect(this, &PipelineHandlerC3ISP::statsProcessed); + data->ipa_->paramsComputed.connect(this, &PipelineHandlerC3ISP::paramsComputed); + + std::set streams{ &data->viewStream, &data->stillStream, &data->videoStream }; + + std::shared_ptr camera = Camera::create(std::move(data), sensor->name(), streams); + + registerCamera(std::move(camera)); + + return true; +} + +bool PipelineHandlerC3ISP::match(DeviceEnumerator *enumerator) +{ + const MediaPad *ispSink; + + DeviceMatch dm("c3-isp"); + dm.add("c3-mipi-csi2"); + dm.add("c3-mipi-adapter"); + dm.add("c3-isp-core"); + + media_ = acquireMediaDevice(enumerator, dm); + if (!media_) + return false; + + isp_ = V4L2Subdevice::fromEntityName(media_, "c3-isp-core"); + if (isp_->open() < 0) + return false; + + stats_ = V4L2VideoDevice::fromEntityName(media_, "c3-isp-stats"); + if (stats_->open() < 0) + return false; + + params_ = V4L2VideoDevice::fromEntityName(media_, "c3-isp-params"); + if (params_->open() < 0) + return false; + + for (unsigned int i = C3ISPVIEW; i < C3ISPNumPipes; i++) { + C3ISPPipe *ispPipe = &pipes_[i]; + + std::ostringstream resizer; + resizer << "c3-isp-resizer" << i; + ispPipe->resizer = V4L2Subdevice::fromEntityName(media_, resizer.str()); + if (ispPipe->resizer->open() < 0) + return false; + + std::ostringstream capture; + capture << "c3-isp-cap" << i; + ispPipe->cap = V4L2VideoDevice::fromEntityName(media_, capture.str()); + if (ispPipe->cap->open() < 0) + return false; + + ispPipe->cap->bufferReady.connect(this, &PipelineHandlerC3ISP::imageBufferReady); + } + + stats_->bufferReady.connect(this, &PipelineHandlerC3ISP::statsBufferReady); + params_->bufferReady.connect(this, &PipelineHandlerC3ISP::paramsBufferReady); + + ispSink = isp_->entity()->getPadByIndex(0); + if (!ispSink || ispSink->links().empty()) + return false; + + if (!createCamera(ispSink->links()[0])) { + LOG(C3ISP, Error) << "Failed to create camera"; + return false; + } + + return true; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerC3ISP, "c3isp") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/c3-isp/meson.build b/src/libcamera/pipeline/c3-isp/meson.build new file mode 100644 index 00000000..5f8b23f1 --- /dev/null +++ b/src/libcamera/pipeline/c3-isp/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'c3-isp.cpp' +]) From patchwork Thu Feb 27 10:57:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22886 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A59BABF415 for ; Thu, 27 Feb 2025 10:57:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CA0068753; Thu, 27 Feb 2025 11:57:48 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0517F68754 for ; Thu, 27 Feb 2025 11:57:41 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:57:37 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 03/11] include: linux: Add c3-isp-config.h Date: Thu, 27 Feb 2025 18:57:25 +0800 Message-ID: <20250227105733.187611-4-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the header file describing the C3 ISP's 3A statistics and parameters structures so that they can be used by the new IPA module. The header is a temporary addition and has been taken from the v6 of the patchset for the C3 ISP driver submitted to linux-media, found at the link below: https://lore.kernel.org/linux-media/20250227-c3isp-v6-7-f72e19084d0d@amlogic.com/ Signed-off-by: Keke Li --- include/linux/c3-isp-config.h | 564 ++++++++++++++++++++++++++++++++++ 1 file changed, 564 insertions(+) create mode 100644 include/linux/c3-isp-config.h diff --git a/include/linux/c3-isp-config.h b/include/linux/c3-isp-config.h new file mode 100644 index 00000000..ed085ea6 --- /dev/null +++ b/include/linux/c3-isp-config.h @@ -0,0 +1,564 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2024 Amlogic, Inc. All rights reserved + */ + +#ifndef _UAPI_C3_ISP_CONFIG_H_ +#define _UAPI_C3_ISP_CONFIG_H_ + +#include + +/* + * Frames are split into zones of almost equal width and height - a zone is a + * rectangular tile of a frame. The metering blocks within the ISP collect + * aggregated statistics per zone. + */ +#define C3_ISP_AE_MAX_ZONES (17 * 15) +#define C3_ISP_AF_MAX_ZONES (17 * 15) +#define C3_ISP_AWB_MAX_ZONES (32 * 24) + +/* The maximum number of point on the diagonal of the frame for statistics */ +#define C3_ISP_AE_MAX_PT_NUM 18 +#define C3_ISP_AF_MAX_PT_NUM 18 +#define C3_ISP_AWB_MAX_PT_NUM 33 + +/** + * struct c3_isp_awb_zone_stats - AWB statistics of a zone + * + * AWB zone stats is aligned with 8 bytes + * + * @rg: the ratio of R / G in a zone + * @bg: the ratio of B / G in a zone + * @pixel_sum: the total number of pixels used in a zone + */ +struct c3_isp_awb_zone_stats { + __u16 rg; + __u16 bg; + __u32 pixel_sum; +}; + +/** + * struct c3_isp_awb_stats - Auto white balance statistics information. + * + * AWB statistical information of all zones. + * + * @stats: array of auto white balance statistics + */ +struct c3_isp_awb_stats { + struct c3_isp_awb_zone_stats stats[C3_ISP_AWB_MAX_ZONES]; +} __attribute__((aligned(16))); + +/** + * struct c3_isp_ae_zone_stats - AE statistics of a zone + * + * AE zone stats is aligned with 8 bytes. + * This is a 5-bin histogram and the total sum is normalized to 0xffff. + * So hist2 = 0xffff - (hist0 + hist1 + hist3 + hist4) + * + * @hist0: the global normalized pixel count for bin 0 + * @hist1: the global normalized pixel count for bin 1 + * @hist3: the global normalized pixel count for bin 3 + * @hist4: the global normalized pixel count for bin 4 + */ +struct c3_isp_ae_zone_stats { + __u16 hist0; + __u16 hist1; + __u16 hist3; + __u16 hist4; +}; + +/** + * struct c3_isp_ae_stats - Exposure statistics information + * + * AE statistical information consists of all blocks information and a 1024-bin + * histogram. + * + * @stats: array of auto exposure block statistics + * @reserved: undefined buffer space + * @hist: a 1024-bin histogram for the entire image + */ +struct c3_isp_ae_stats { + struct c3_isp_ae_zone_stats stats[C3_ISP_AE_MAX_ZONES]; + __u32 reserved[2]; + __u32 hist[1024]; +} __attribute__((aligned(16))); + +/** + * struct c3_isp_af_zone_stats - AF statistics of a zone + * + * AF zone stats is aligned with 8 bytes. + * The zonal accumulated contrast metrics are stored in floating point format + * with 16 bits mantissa and 5 or 6 bits exponent. Apart from contrast metrics + * we accumulate squared image and quartic image data over the zone. + * + * @i2_mat: the mantissa of zonal squared image pixel sum + * @i4_mat: the mantissa of zonal quartic image pixel sum + * @e4_mat: the mantissa of zonal multi-directional quartic edge sum + * @e4_exp: the exponent of zonal multi-directional quartic edge sum + * @i2_exp: the exponent of zonal squared image pixel sum + * @i4_exp: the exponent of zonal quartic image pixel sum + */ +struct c3_isp_af_zone_stats { + __u16 i2_mat; + __u16 i4_mat; + __u16 e4_mat; + __u16 e4_exp : 5; + __u16 i2_exp : 5; + __u16 i4_exp : 6; +}; + +/** + * struct c3_isp_af_stats - Auto Focus statistics information + * + * AF statistical information of each zone + * + * @stats: array of auto focus block statistics + * @reserved: undefined buffer space + */ +struct c3_isp_af_stats { + struct c3_isp_af_zone_stats stats[C3_ISP_AF_MAX_ZONES]; + __u32 reserved[2]; +} __attribute__((aligned(16))); + +/** + * struct c3_isp_stats_info - V4L2_META_FMT_C3ISP_STATS + * + * Contains ISP statistics + * + * @awb: auto white balance stats + * @ae: auto exposure stats + * @af: auto focus stats + */ +struct c3_isp_stats_info { + struct c3_isp_awb_stats awb; + struct c3_isp_ae_stats ae; + struct c3_isp_af_stats af; +}; + +/** + * enum c3_isp_params_buffer_version - C3 ISP parameters block versioning + * + * @C3_ISP_PARAMS_BUFFER_V0: First version of C3 ISP parameters block + */ +enum c3_isp_params_buffer_version { + C3_ISP_PARAMS_BUFFER_V0, +}; + +/** + * enum c3_isp_params_block_type - Enumeration of C3 ISP parameter blocks + * + * Each block configures a specific processing block of the C3 ISP. + * The block type allows the driver to correctly interpret the parameters block + * data. + * + * @C3_ISP_PARAMS_BLOCK_AWB_GAINS: White balance gains + * @C3_ISP_PARAMS_BLOCK_AWB_CONFIG: AWB statistic format configuration for all + * blocks that control how stats are generated + * @C3_ISP_PARAMS_BLOCK_AE_CONFIG: AE statistic format configuration for all + * blocks that control how stats are generated + * @C3_ISP_PARAMS_BLOCK_AF_CONFIG: AF statistic format configuration for all + * blocks that control how stats are generated + * @C3_ISP_PARAMS_BLOCK_PST_GAMMA: post gamma parameters + * @C3_ISP_PARAMS_BLOCK_CCM: Color correction matrix parameters + * @C3_ISP_PARAMS_BLOCK_CSC: Color space conversion parameters + * @C3_ISP_PARAMS_BLOCK_BLC: Black level correction parameters + * @C3_ISP_PARAMS_BLOCK_SENTINEL: First non-valid block index + */ +enum c3_isp_params_block_type { + C3_ISP_PARAMS_BLOCK_AWB_GAINS, + C3_ISP_PARAMS_BLOCK_AWB_CONFIG, + C3_ISP_PARAMS_BLOCK_AE_CONFIG, + C3_ISP_PARAMS_BLOCK_AF_CONFIG, + C3_ISP_PARAMS_BLOCK_PST_GAMMA, + C3_ISP_PARAMS_BLOCK_CCM, + C3_ISP_PARAMS_BLOCK_CSC, + C3_ISP_PARAMS_BLOCK_BLC, + C3_ISP_PARAMS_BLOCK_SENTINEL +}; + +#define C3_ISP_PARAMS_BLOCK_FL_DISABLE (1U << 0) +#define C3_ISP_PARAMS_BLOCK_FL_ENABLE (1U << 1) + +/** + * struct c3_isp_params_block_header - C3 ISP parameter block header + * + * This structure represents the common part of all the ISP configuration + * blocks. Each parameters block shall embed an instance of this structure type + * as its first member, followed by the block-specific configuration data. The + * driver inspects this common header to discern the block type and its size and + * properly handle the block content by casting it to the correct block-specific + * type. + * + * The @type field is one of the values enumerated by + * :c:type:`c3_isp_params_block_type` and specifies how the data should be + * interpreted by the driver. The @size field specifies the size of the + * parameters block and is used by the driver for validation purposes. The + * @flags field is a bitmask of per-block flags C3_ISP_PARAMS_FL*. + * + * When userspace wants to disable an ISP block the + * C3_ISP_PARAMS_BLOCK_FL_DISABLED bit should be set in the @flags field. In + * this case userspace may optionally omit the remainder of the configuration + * block, which will be ignored by the driver. + * + * When a new configuration of an ISP block needs to be applied userspace + * shall fully populate the ISP block and omit setting the + * C3_ISP_PARAMS_BLOCK_FL_DISABLED bit in the @flags field. + * + * Userspace is responsible for correctly populating the parameters block header + * fields (@type, @flags and @size) and the block-specific parameters. + * + * For example: + * + * .. code-block:: c + * + * void populate_pst_gamma(struct c3_isp_params_block_header *block) { + * struct c3_isp_params_pst_gamma *gamma = + * (struct c3_isp_params_pst_gamma *)block; + * + * gamma->header.type = C3_ISP_PARAMS_BLOCK_PST_GAMMA; + * gamma->header.flags = C3_ISP_PARAMS_BLOCK_FL_ENABLE; + * gamma->header.size = sizeof(*gamma); + * + * for (unsigned int i = 0; i < 129; i++) + * gamma->pst_gamma_lut[i] = i; + * } + * + * @type: The parameters block type from :c:type:`c3_isp_params_block_type` + * @flags: A bitmask of block flags + * @size: Size (in bytes) of the parameters block, including this header + */ +struct c3_isp_params_block_header { + __u16 type; + __u16 flags; + __u32 size; +}; + +/** + * struct c3_isp_params_awb_gains - Gains for auto-white balance + * + * This struct allows users to configure the gains for white balance. + * There are four gain settings corresponding to each colour channel in + * the bayer domain. All of the gains are stored in Q4.8 format. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_AWB_GAINS + * from :c:type:`c3_isp_params_block_type` + * + * @header: The C3 ISP parameters block header + * @gr_gain: Multiplier for Gr channel (Q4.8 format) + * @r_gain: Multiplier for R channel (Q4.8 format) + * @b_gain: Multiplier for B channel (Q4.8 format) + * @gb_gain: Multiplier for Gb channel (Q4.8 format) + */ +struct c3_isp_params_awb_gains { + struct c3_isp_params_block_header header; + __u16 gr_gain; + __u16 r_gain; + __u16 b_gain; + __u16 gb_gain; +} __attribute__((aligned(8))); + +/** + * enum c3_isp_params_awb_tap_points - Tap points for the AWB statistics + * @C3_ISP_AWB_STATS_TAP_OFE: immediately after the optical frontend block + * @C3_ISP_AWB_STATS_TAP_GE: immediately after the green equal block + * @C3_ISP_AWB_STATS_TAP_BEFORE_WB: immediately before the white balance block + * @C3_ISP_AWB_STATS_TAP_AFTER_WB: immediately after the white balance block + */ +enum c3_isp_params_awb_tap_points { + C3_ISP_AWB_STATS_TAP_OFE = 0, + C3_ISP_AWB_STATS_TAP_GE, + C3_ISP_AWB_STATS_TAP_BEFORE_WB, + C3_ISP_AWB_STATS_TAP_AFTER_WB, +}; + +/** + * struct c3_isp_params_awb_config - Stats settings for auto-white balance + * + * This struct allows the configuration of the statistics generated for auto + * white balance. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_AWB_CONFIG + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @tap_point: the tap point from enum c3_isp_params_awb_tap_point + * @satur_vald: AWB statistic over saturation control + * value: 0: disable, 1: enable + * @horiz_zones_num: active number of hotizontal zones [0..32] + * @vert_zones_num: active number of vertical zones [0..24] + * @rg_min: minimum R/G ratio (Q4.8 format) + * @rg_max: maximum R/G ratio (Q4.8 format) + * @bg_min: minimum B/G ratio (Q4.8 format) + * @bg_max: maximum B/G ratio (Q4.8 format) + * @rg_low: R/G ratio trim low (Q4.8 format) + * @rg_high: R/G ratio trim hight (Q4.8 format) + * @bg_low: B/G ratio trim low (Q4.8 format) + * @bg_high: B/G ratio trim high (Q4.8 format) + * @zone_weight: array of weights for AWB statistics zones [0..15] + * @horiz_coord: the horizontal coordinate of points on the diagonal [0..2888] + * @vert_coord: the vertical coordinate of points on the diagonal [0..2240] + */ +struct c3_isp_params_awb_config { + struct c3_isp_params_block_header header; + __u8 tap_point; + __u8 satur_vald; + __u8 horiz_zones_num; + __u8 vert_zones_num; + __u16 rg_min; + __u16 rg_max; + __u16 bg_min; + __u16 bg_max; + __u16 rg_low; + __u16 rg_high; + __u16 bg_low; + __u16 bg_high; + __u8 zone_weight[C3_ISP_AWB_MAX_ZONES]; + __u16 horiz_coord[C3_ISP_AWB_MAX_PT_NUM]; + __u16 vert_coord[C3_ISP_AWB_MAX_PT_NUM]; +} __attribute__((aligned(8))); + +/** + * enum c3_isp_params_ae_tap_points - Tap points for the AE statistics + * @C3_ISP_AE_STATS_TAP_GE: immediately after the green equal block + * @C3_ISP_AE_STATS_TAP_MLS: immediately after the mesh lens shading block + */ +enum c3_isp_params_ae_tap_points { + C3_ISP_AE_STATS_TAP_GE = 0, + C3_ISP_AE_STATS_TAP_MLS, +}; + +/** + * struct c3_isp_params_ae_config - Stats settings for auto-exposure + * + * This struct allows the configuration of the statistics generated for + * auto exposure. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_AE_CONFIG + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @horiz_zones_num: active number of horizontal zones [0..17] + * @vert_zones_num: active number of vertical zones [0..15] + * @tap_point: the tap point from enum c3_isp_params_ae_tap_point + * @zone_weight: array of weights for AE statistics zones [0..15] + * @horiz_coord: the horizontal coordinate of points on the diagonal [0..2888] + * @vert_coord: the vertical coordinate of points on the diagonal [0..2240] + * @reserved: applications must zero this array + */ +struct c3_isp_params_ae_config { + struct c3_isp_params_block_header header; + __u8 tap_point; + __u8 horiz_zones_num; + __u8 vert_zones_num; + __u8 zone_weight[C3_ISP_AE_MAX_ZONES]; + __u16 horiz_coord[C3_ISP_AE_MAX_PT_NUM]; + __u16 vert_coord[C3_ISP_AE_MAX_PT_NUM]; + __u16 reserved[3]; +} __attribute__((aligned(8))); + +/** + * enum c3_isp_params_af_tap_points - Tap points for the AF statistics + * @C3_ISP_AF_STATS_TAP_SNR: immediately after the spatial noise reduce block + * @C3_ISP_AF_STATS_TAP_DMS: immediately after the demosaic block + */ +enum c3_isp_params_af_tap_points { + C3_ISP_AF_STATS_TAP_SNR = 0, + C3_ISP_AF_STATS_TAP_DMS, +}; + +/** + * struct c3_isp_params_af_config - Stats settings for auto-focus + * + * This struct allows the configuration of the statistics generated for + * auto focus. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_AF_CONFIG + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @tap_point: the tap point from enum c3_isp_params_af_tap_point + * @horiz_zones_num: active number of hotizontal zones [0..17] + * @vert_zones_num: active number of vertical zones [0..15] + * @reserved: applications must zero this array + * @horiz_coord: the horizontal coordinate of points on the diagonal [0..2888] + * @vert_coord: the vertical coordinate of points on the diagonal [0..2240] + */ +struct c3_isp_params_af_config { + struct c3_isp_params_block_header header; + __u8 tap_point; + __u8 horiz_zones_num; + __u8 vert_zones_num; + __u8 reserved[5]; + __u16 horiz_coord[C3_ISP_AF_MAX_PT_NUM]; + __u16 vert_coord[C3_ISP_AF_MAX_PT_NUM]; +} __attribute__((aligned(8))); + +/** + * struct c3_isp_params_pst_gamma - Post gamma configuration + * + * This struct allows the configuration of the look up table for + * post gamma. The gamma curve consists of 129 points, so need to + * set lut[129]. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_PST_GAMMA + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @lut: lookup table for P-Stitch gamma [0..1023] + * @reserved: applications must zero this array + */ +struct c3_isp_params_pst_gamma { + struct c3_isp_params_block_header header; + __u16 lut[129]; + __u16 reserved[3]; +} __attribute__((aligned(8))); + +/** + * struct c3_isp_params_ccm - ISP CCM configuration + * + * This struct allows the configuration of the matrix for + * color correction. The matrix consists of 3 x 3 points, + * so need to set matrix[3][3]. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_CCM + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @matrix: a 3 x 3 matrix used for color correction, + * the value of matrix[x][y] is orig_value x 256. [-4096..4095] + * @reserved: applications must zero this array + */ +struct c3_isp_params_ccm { + struct c3_isp_params_block_header header; + __s16 matrix[3][3]; + __u16 reserved[3]; +} __attribute__((aligned(8))); + +/** + * struct c3_isp_params_csc - ISP Color Space Conversion configuration + * + * This struct allows the configuration of the matrix for color space + * conversion. The matrix consists of 3 x 3 points, so need to set matrix[3][3]. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_CSC + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @matrix: a 3x3 matrix used for the color space conversion, + * the value of matrix[x][y] is orig_value x 256. [-4096..4095] + * @reserved: applications must zero this array + */ +struct c3_isp_params_csc { + struct c3_isp_params_block_header header; + __s16 matrix[3][3]; + __u16 reserved[3]; +} __attribute__((aligned(8))); + +/** + * struct c3_isp_params_blc - ISP Black Level Correction configuration + * + * This struct allows the configuration of the block level offset for each + * color channel. + * + * header.type should be set to C3_ISP_PARAMS_BLOCK_BLC + * from :c:type:`c3_isp_params_block_type` + * + * @header: the C3 ISP parameters block header + * @gr_ofst: Gr blc offset (Q4.12 format) + * @r_ofst: R blc offset (Q4.12 format) + * @b_ofst: B blc offset (Q4.12 format) + * @gb_ofst: Gb blc offset(Q4.12 format) + */ +struct c3_isp_params_blc { + struct c3_isp_params_block_header header; + __u16 gr_ofst; + __u16 r_ofst; + __u16 b_ofst; + __u16 gb_ofst; +}; + +/** + * define C3_ISP_PARAMS_MAX_SIZE - Maximum size of all C3 ISP Parameters + * + * Though the parameters for the C3 ISP are passed as optional blocks, the + * driver still needs to know the absolute maximum size so that it can allocate + * a buffer sized appropriately to accommodate userspace attempting to set all + * possible parameters in a single frame. + */ +#define C3_ISP_PARAMS_MAX_SIZE \ + (sizeof(struct c3_isp_params_awb_gains) + \ + sizeof(struct c3_isp_params_awb_config) + \ + sizeof(struct c3_isp_params_ae_config) + \ + sizeof(struct c3_isp_params_af_config) + \ + sizeof(struct c3_isp_params_pst_gamma) + \ + sizeof(struct c3_isp_params_ccm) + \ + sizeof(struct c3_isp_params_csc) + \ + sizeof(struct c3_isp_params_blc)) + +/** + * struct c3_isp_params_cfg - C3 ISP configuration parameters + * + * This struct contains the configuration parameters of the C3 ISP + * algorithms, serialized by userspace into an opaque data buffer. Each + * configuration parameter block is represented by a block-specific structure + * which contains a :c:type:`c3_isp_param_block_header` entry as first + * member. Userspace populates the @data buffer with configuration parameters + * for the blocks that it intends to configure. As a consequence, the data + * buffer effective size changes according to the number of ISP blocks that + * userspace intends to configure. + * + * The parameters buffer is versioned by the @version field to allow modifying + * and extending its definition. Userspace should populate the @version field to + * inform the driver about the version it intends to use. The driver will parse + * and handle the @data buffer according to the data layout specific to the + * indicated revision and return an error if the desired revision is not + * supported. + * + * For each ISP block that userspace wants to configure, a block-specific + * structure is appended to the @data buffer, one after the other without gaps + * in between nor overlaps. Userspace shall populate the @total_size field with + * the effective size, in bytes, of the @data buffer. + * + * The expected memory layout of the parameters buffer is:: + * + * +-------------------- struct c3_isp_params_cfg ---- ------------------+ + * | version = C3_ISP_PARAM_BUFFER_V0; | + * | data_size = sizeof(struct c3_isp_params_awb_gains) + | + * | sizeof(struct c3_isp_params_awb_config); | + * | +------------------------- data ---------------------------------+ | + * | | +------------ struct c3_isp_params_awb_gains) ------------------+ | + * | | | +--------- struct c3_isp_params_block_header header -----+ | | | + * | | | | type = C3_ISP_PARAMS_BLOCK_AWB_GAINS; | | | | + * | | | | flags = C3_ISP_PARAMS_BLOCK_FL_NONE; | | | | + * | | | | size = sizeof(struct c3_isp_params_awb_gains); | | | | + * | | | +---------------------------------------------------------+ | | | + * | | | gr_gain = ...; | | | + * | | | r_gain = ...; | | | + * | | | b_gain = ...; | | | + * | | | gb_gain = ...; | | | + * | | +------------------ struct c3_isp_params_awb_config ----------+ | | + * | | | +---------- struct c3_isp_param_block_header header ------+ | | | + * | | | | type = C3_ISP_PARAMS_BLOCK_AWB_CONFIG; | | | | + * | | | | flags = C3_ISP_PARAMS_BLOCK_FL_NONE; | | | | + * | | | | size = sizeof(struct c3_isp_params_awb_config) | | | | + * | | | +---------------------------------------------------------+ | | | + * | | | tap_point = ...; | | | + * | | | satur_vald = ...; | | | + * | | | horiz_zones_num = ...; | | | + * | | | vert_zones_num = ...; | | | + * | | +-------------------------------------------------------------+ | | + * | +-----------------------------------------------------------------+ | + * +---------------------------------------------------------------------+ + * + * @version: The C3 ISP parameters buffer version + * @data_size: The C3 ISP configuration data effective size, excluding this + * header + * @data: The C3 ISP configuration blocks data + */ +struct c3_isp_params_cfg { + __u32 version; + __u32 data_size; + __u8 data[C3_ISP_PARAMS_MAX_SIZE]; +}; + +#endif From patchwork Thu Feb 27 10:57:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22887 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7DC08BF415 for ; Thu, 27 Feb 2025 10:57:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B3396875B; Thu, 27 Feb 2025 11:57:50 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A513C686E7 for ; Thu, 27 Feb 2025 11:57:42 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:57:38 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 04/11] ipa: c3-isp: Add C3 ISP IPA module Date: Thu, 27 Feb 2025 18:57:26 +0800 Message-ID: <20250227105733.187611-5-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a barebones IPA module for the C3 ISP. In this initial implementation only basic interfaces are implemented. Signed-off-by: Keke Li --- meson_options.txt | 2 +- src/ipa/c3-isp/algorithms/algorithm.h | 28 ++ src/ipa/c3-isp/algorithms/meson.build | 5 + src/ipa/c3-isp/c3-isp.cpp | 370 ++++++++++++++++++++++++++ src/ipa/c3-isp/data/meson.build | 9 + src/ipa/c3-isp/ipa_context.cpp | 101 +++++++ src/ipa/c3-isp/ipa_context.h | 87 ++++++ src/ipa/c3-isp/meson.build | 32 +++ src/ipa/c3-isp/module.h | 28 ++ src/ipa/c3-isp/params.cpp | 127 +++++++++ src/ipa/c3-isp/params.h | 133 +++++++++ 11 files changed, 921 insertions(+), 1 deletion(-) create mode 100644 src/ipa/c3-isp/algorithms/algorithm.h create mode 100644 src/ipa/c3-isp/algorithms/meson.build create mode 100644 src/ipa/c3-isp/c3-isp.cpp create mode 100644 src/ipa/c3-isp/data/meson.build create mode 100644 src/ipa/c3-isp/ipa_context.cpp create mode 100644 src/ipa/c3-isp/ipa_context.h create mode 100644 src/ipa/c3-isp/meson.build create mode 100644 src/ipa/c3-isp/module.h create mode 100644 src/ipa/c3-isp/params.cpp create mode 100644 src/ipa/c3-isp/params.h diff --git a/meson_options.txt b/meson_options.txt index 998a4463..7c666d34 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -32,7 +32,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/vc4', 'simple', 'vimc'], + choices : ['c3-isp', 'ipu3', 'mali-c55', 'rkisp1', 'rpi/vc4', 'simple', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', diff --git a/src/ipa/c3-isp/algorithms/algorithm.h b/src/ipa/c3-isp/algorithms/algorithm.h new file mode 100644 index 00000000..68c311d7 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/algorithm.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP control algorithm interface + */ + +#pragma once + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::c3isp { + +class Algorithm : public libcamera::ipa::Algorithm +{ +public: + Algorithm() + { + } +}; + +} /* namespace ipa::c3isp */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build new file mode 100644 index 00000000..1e00af33 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +c3isp_ipa_algorithms = files([ + +]) diff --git a/src/ipa/c3-isp/c3-isp.cpp b/src/ipa/c3-isp/c3-isp.cpp new file mode 100644 index 00000000..abb56d15 --- /dev/null +++ b/src/ipa/c3-isp/c3-isp.cpp @@ -0,0 +1,370 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * c3-isp.cpp - Amlogic Image Processing Algorithms + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" +#include "libipa/camera_sensor_helper.h" + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPAC3ISP) + +using namespace std::literals::chrono_literals; + +namespace ipa::c3isp { + +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPAC3ISP : public IPAC3ISPInterface, public Module +{ +public: + IPAC3ISP(); + + int init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers, bool readOnly) override; + void unmapBuffers(const std::vector &buffers) override; + + void queueRequest(const uint32_t request, const ControlList &controls) override; + void computeParams(const uint32_t request, const uint32_t bufferId) override; + void processStats(const uint32_t request, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateSessionConfiguration(const IPACameraSensorInfo &info, + const ControlInfoMap &sensorControls); + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); + void setControls(); + + std::map buffers_; + + ControlInfoMap sensorControls_; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +IPAC3ISP::IPAC3ISP() + : context_(kMaxFrameContexts) +{ +} + +std::string IPAC3ISP::logPrefix() const +{ + return "c3isp"; +} + +int IPAC3ISP::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!camHelper_) { + LOG(IPAC3ISP, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPAC3ISP, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + if (!data->contains("algorithms")) { + LOG(IPAC3ISP, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + updateControls(ipaConfig.sensorInfo, ipaConfig.sensorControls, ipaControls); + + return 0; +} + +int IPAC3ISP::start() +{ + return 0; +} + +void IPAC3ISP::stop() +{ + context_.frameContexts.clear(); +} + +void IPAC3ISP::updateSessionConfiguration(const IPACameraSensorInfo &info, + const ControlInfoMap &sensorControls) +{ + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get(); + int32_t maxExposure = v4l2Exposure.max().get(); + int32_t defExposure = v4l2Exposure.def().get(); + + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + int32_t minGain = v4l2Gain.min().get(); + int32_t maxGain = v4l2Gain.max().get(); + + context_.configuration.sensor.lineDuration = info.minLineLength * 1.0s / info.pixelRate; + context_.configuration.agc.minShutterSpeed = minExposure * context_.configuration.sensor.lineDuration; + context_.configuration.agc.maxShutterSpeed = maxExposure * context_.configuration.sensor.lineDuration; + context_.configuration.agc.defaultExposure = defExposure; + context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); + context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); + + context_.configuration.sensor.size = info.outputSize; + + if (camHelper_->blackLevel().has_value()) + /* + * The black level from CameraSensorHelper is a 16-bit value. + * The C3 ISP expects 20-bit settings, so we shift it to the + * appropriate width + */ + context_.configuration.sensor.blackLevel = camHelper_->blackLevel().value() << 4; +} + +void IPAC3ISP::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap; + + /* Compute the frame duration limits. */ + const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get(); + uint32_t lineLength = sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; + std::array frameHeights{ + v4l2VBlank.min().get() + sensorInfo.outputSize.height, + v4l2VBlank.max().get() + sensorInfo.outputSize.height, + v4l2VBlank.def().get() + sensorInfo.outputSize.height, + }; + + std::array frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + } + + ctrlMap[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], + frameDurations[1], + frameDurations[2]); + + /* Compute the exposure time limits */ + double lineDuration = context_.configuration.sensor.lineDuration.get(); + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap[&controls::ExposureTime] = ControlInfo(minExposure, maxExposure, defExposure); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = camHelper_->gain(v4l2Gain.min().get()); + float maxGain = camHelper_->gain(v4l2Gain.max().get()); + float defGain = camHelper_->gain(v4l2Gain.def().get()); + ctrlMap[&controls::AnalogueGain] = ControlInfo(minGain, maxGain, defGain); + + /* Merge the cotrols */ + ctrlMap.merge(context_.ctrlMap); + + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +int IPAC3ISP::configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + sensorControls_ = ipaConfig.sensorControls; + + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + + updateSessionConfiguration(info, ipaConfig.sensorControls); + + updateControls(info, ipaConfig.sensorControls, ipaControls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPAC3ISP::mapBuffers(const std::vector &buffers, bool readOnly) +{ + for (const IPABuffer &buffer : buffers) { + const FrameBuffer fb(buffer.planes); + buffers_.emplace( + buffer.id, + MappedFrameBuffer( + &fb, + readOnly ? MappedFrameBuffer::MapFlag::Read + : MappedFrameBuffer::MapFlag::ReadWrite)); + } +} + +void IPAC3ISP::unmapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto it = buffers_.find(buffer.id); + if (it == buffers_.end()) + continue; + + buffers_.erase(buffer.id); + } +} + +void IPAC3ISP::queueRequest(const uint32_t request, const ControlList &controls) +{ + IPAFrameContext &frameContext = context_.frameContexts.alloc(request); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + algo->queueRequest(context_, request, frameContext, controls); + } +} + +void IPAC3ISP::computeParams(const uint32_t request, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(request); + + C3ISPParams params(buffers_.at(bufferId).planes()[0]); + + for (auto const &algo : algorithms()) + algo->prepare(context_, request, frameContext, ¶ms); + + paramsComputed.emit(request, params.size()); +} + +void IPAC3ISP::processStats(const uint32_t request, const uint32_t bufferId, + const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(request); + const c3_isp_stats_info *stats = nullptr; + + stats = reinterpret_cast( + buffers_.at(bufferId).planes()[0].data()); + + frameContext.agc.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.agc.sensorGain = + camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + + ControlList metadata(controls::controls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + algo->process(context_, request, frameContext, stats, metadata); + } + + setControls(); + + statsProcessed.emit(request, metadata); +} + +void IPAC3ISP::setControls() +{ + IPAActiveState &activeState = context_.activeState; + uint32_t exposure; + uint32_t gain; + + if (activeState.agc.autoEnabled) { + exposure = activeState.agc.automatic.exposure; + gain = camHelper_->gainCode(activeState.agc.automatic.sensorGain); + } else { + exposure = activeState.agc.manual.exposure; + gain = camHelper_->gainCode(activeState.agc.manual.sensorGain); + } + + ControlList ctrls(sensorControls_); + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); + + setSensorControls.emit(ctrls); +} + +} /* namespace ipa::c3isp */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "c3isp", + "c3isp", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::c3isp::IPAC3ISP(); +} +} + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/data/meson.build b/src/ipa/c3-isp/data/meson.build new file mode 100644 index 00000000..75e44bac --- /dev/null +++ b/src/ipa/c3-isp/data/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'c3isp', + install_tag : 'runtime') diff --git a/src/ipa/c3-isp/ipa_context.cpp b/src/ipa/c3-isp/ipa_context.cpp new file mode 100644 index 00000000..8f69bbfb --- /dev/null +++ b/src/ipa/c3-isp/ipa_context.cpp @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * C3ISP IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::c3isp { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAActiveState + * \brief Active state for algorithms + * + * The active state contains all algorithm-specific data that needs to be + * maintained by algorithms across frames. Unlike the session configuration, + * the active state is mutable and constantly updated by algorithms. The active + * state is accessible through the IPAContext structure. + * + * The active state stores two distinct categories of information: + * + * - The consolidated value of all algorithm controls. Requests passed to + * the queueRequest() function store values for controls that the + * application wants to modify for that particular frame, and the + * queueRequest() function updates the active state with those values. + * The active state thus contains a consolidated view of the value of all + * controls handled by the algorithm. + * + * - The value of parameters computed by the algorithm when running in auto + * mode. Algorithms running in auto mode compute new parameters every + * time statistics buffers are received (either synchronously, or + * possibly in a background thread). The latest computed value of those + * parameters is stored in the active state in the process() function. + * + * Each of the members in the active state belongs to a specific algorithm. A + * member may be read by any algorithm, but shall only be written by its owner. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores two distinct categories of information: + * + * - The value of the controls to be applied to the frame. These values are + * typically set in the queueRequest() function, from the consolidated + * control values stored in the active state. The frame context thus stores + * values for all controls related to the algorithm, not limited to the + * controls specified in the corresponding request, but consolidated from all + * requests that have been queued so far. + * + * For controls that can be set manually or computed by an algorithm + * (depending on the algorithm operation mode), such as for instance the + * colour gains for the AWB algorithm, the control value will be stored in + * the frame context in the queueRequest() function only when operating in + * manual mode. When operating in auto mode, the values are computed by the + * algorithm in process(), stored in the active state, and copied to the + * frame context in prepare(), just before being stored in the ISP parameters + * buffer. + * + * The queueRequest() function can also store ancillary data in the frame + * context, such as flags to indicate if (and what) control values have + * changed compared to the previous request. + * + * - Status information computed by the algorithm for a frame. For instance, + * the colour temperature estimated by the AWB algorithm from ISP statistics + * calculated on a frame is stored in the frame context for that frame in + * the process() function. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::activeState + * \brief The IPA active state, storing the latest state for all algorithms + * + * \var IPAContext::frameContexts + * \brief Ring buffer of per-frame contexts + */ + +} /* namespace libcamera::ipa::c3isp */ diff --git a/src/ipa/c3-isp/ipa_context.h b/src/ipa/c3-isp/ipa_context.h new file mode 100644 index 00000000..227d657a --- /dev/null +++ b/src/ipa/c3-isp/ipa_context.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * C3ISP IPA Context + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::c3isp { + +struct IPASessionConfiguration { + struct { + utils::Duration minShutterSpeed; + utils::Duration maxShutterSpeed; + uint32_t defaultExposure; + double minAnalogueGain; + double maxAnalogueGain; + } agc; + + struct { + utils::Duration lineDuration; + uint32_t blackLevel; + Size size; + } sensor; +}; + +struct IPAActiveState { + struct { + struct { + uint32_t exposure; + double sensorGain; + double ispGain; + } automatic; + struct { + uint32_t exposure; + double sensorGain; + double ispGain; + } manual; + bool autoEnabled; + uint32_t constraintMode; + uint32_t exposureMode; + } agc; + + struct { + double rGain; + double bGain; + uint32_t temperatureK; + } awb; +}; + +struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double sensorGain; + } agc; + + struct { + double rGain; + double bGain; + } awb; +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; +}; + +} /* namespace ipa::c3isp */ + +} /* namespace libcamera*/ diff --git a/src/ipa/c3-isp/meson.build b/src/ipa/c3-isp/meson.build new file mode 100644 index 00000000..fa5c6be0 --- /dev/null +++ b/src/ipa/c3-isp/meson.build @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') +subdir('data') + +ipa_name = 'ipa_c3isp' + +c3isp_ipa_sources = files([ + 'ipa_context.cpp', + 'params.cpp', + 'c3-isp.cpp', +]) + +c3isp_ipa_sources += c3isp_ipa_algorithms + +mod = shared_module(ipa_name, c3isp_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/c3-isp/module.h b/src/ipa/c3-isp/module.h new file mode 100644 index 00000000..a1116396 --- /dev/null +++ b/src/ipa/c3-isp/module.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::c3isp { + +using Module = ipa::Module; + +} /* namespace ipa::c3isp */ + +} /* namespace libcamera*/ diff --git a/src/ipa/c3-isp/params.cpp b/src/ipa/c3-isp/params.cpp new file mode 100644 index 00000000..d5c57da3 --- /dev/null +++ b/src/ipa/c3-isp/params.cpp @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * C3ISP ISP Parameters + */ + +#include "params.h" + +#include +#include +#include + +#include +#include + +#include +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(C3ISPParams) + +namespace ipa::c3isp { + +namespace { + +struct BlockTypeInfo { + enum c3_isp_params_block_type type; + size_t size; +}; + +#define C3ISP_BLOCK_TYPE_ENTRY(block, id, type) \ + { \ + BlockType::block, \ + { \ + C3_ISP_PARAMS_BLOCK_##id, \ + sizeof(struct c3_isp_params_##type), \ + } \ + } + +const std::map kBlockTypeInfo = { + C3ISP_BLOCK_TYPE_ENTRY(AWBGains, AWB_GAINS, awb_gains), + C3ISP_BLOCK_TYPE_ENTRY(AWBConfig, AWB_CONFIG, awb_config), + C3ISP_BLOCK_TYPE_ENTRY(AEConfig, AE_CONFIG, ae_config), + C3ISP_BLOCK_TYPE_ENTRY(AFConfig, AF_CONFIG, af_config), + C3ISP_BLOCK_TYPE_ENTRY(PostGamma, PST_GAMMA, pst_gamma), + C3ISP_BLOCK_TYPE_ENTRY(Ccm, CCM, ccm), + C3ISP_BLOCK_TYPE_ENTRY(Csc, CSC, csc), + C3ISP_BLOCK_TYPE_ENTRY(Blc, BLC, blc), +}; + +} /* namespace */ + +C3ISPParamsBlockBase::C3ISPParamsBlockBase(BlockType type, + const Span &data) + : type_(type), data_(data) +{ + header_ = data.subspan(0, sizeof(c3_isp_params_block_header)); +} + +void C3ISPParamsBlockBase::setEnabled(uint16_t flags) +{ + struct c3_isp_params_block_header *header = + reinterpret_cast(header_.data()); + + header->flags = flags; +} + +C3ISPParams::C3ISPParams(Span data) + : data_(data), used_(0) +{ + struct c3_isp_params_cfg *buffer = + reinterpret_cast(data.data()); + + buffer->version = C3_ISP_PARAMS_BUFFER_V0; + buffer->data_size = 0; + + used_ += offsetof(struct c3_isp_params_cfg, data); +} + +Span C3ISPParams::block(BlockType type) +{ + auto infoIt = kBlockTypeInfo.find(type); + if (infoIt == kBlockTypeInfo.end()) { + LOG(C3ISPParams, Error) + << "Invalid parameters type " + << utils::to_underlying(type); + return {}; + } + + const BlockTypeInfo &info = infoIt->second; + + auto cacheIt = blocks_.find(type); + if (cacheIt != blocks_.end()) + return cacheIt->second; + + size_t size = info.size; + if (size > data_.size() - used_) { + LOG(C3ISPParams, Error) + << "No enough remaining space " + << utils::to_underlying(type); + return {}; + } + + Span block = data_.subspan(used_, info.size); + used_ += block.size(); + + struct c3_isp_params_cfg *buffer = + reinterpret_cast(data_.data()); + buffer->data_size += block.size(); + + memset(block.data(), 0, block.size()); + + struct c3_isp_params_block_header *header = + reinterpret_cast(block.data()); + header->type = info.type; + header->size = block.size(); + + blocks_[type] = block; + + return block; +} + +} /* namespace ipa::c3isp */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/params.h b/src/ipa/c3-isp/params.h new file mode 100644 index 00000000..9bb3877b --- /dev/null +++ b/src/ipa/c3-isp/params.h @@ -0,0 +1,133 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic Inc. + * + * C3ISP ISP Parameters + */ + +#pragma once + +#include +#include + +#include + +#include +#include + +namespace libcamera { + +namespace ipa::c3isp { + +enum class BlockType { + AWBGains, + AWBConfig, + AEConfig, + AFConfig, + PostGamma, + Ccm, + Csc, + Blc, +}; + +namespace details { + +template +struct block_type { +}; + +#define C3ISP_DEFINE_BLOCK_TYPE(blocktype, blockStruct) \ + template<> \ + struct block_type { \ + using type = struct c3_isp_params_##blockStruct; \ + }; + +C3ISP_DEFINE_BLOCK_TYPE(AWBGains, awb_gains) +C3ISP_DEFINE_BLOCK_TYPE(AWBConfig, awb_config) +C3ISP_DEFINE_BLOCK_TYPE(AEConfig, ae_config) +C3ISP_DEFINE_BLOCK_TYPE(AFConfig, af_config) +C3ISP_DEFINE_BLOCK_TYPE(PostGamma, pst_gamma) +C3ISP_DEFINE_BLOCK_TYPE(Ccm, ccm) +C3ISP_DEFINE_BLOCK_TYPE(Csc, csc) +C3ISP_DEFINE_BLOCK_TYPE(Blc, blc) + +} /* namespace details */ + +class C3ISPParams; + +class C3ISPParamsBlockBase +{ +public: + C3ISPParamsBlockBase(BlockType type, const Span &data); + + Span data() const { return data_; } + + void setEnabled(uint16_t flags); + +private: + LIBCAMERA_DISABLE_COPY(C3ISPParamsBlockBase) + + BlockType type_; + Span header_; + Span data_; +}; + +template +class C3ISPParamsBlock : public C3ISPParamsBlockBase +{ +public: + using Type = typename details::block_type::type; + + C3ISPParamsBlock(const Span &data) + : C3ISPParamsBlockBase(B, data) + { + } + + const Type *operator->() const + { + return reinterpret_cast(data().data()); + } + + Type *operator->() + { + return reinterpret_cast(data().data()); + } + + const Type &operator*() const & + { + return *reinterpret_cast(data().data()); + } + + const Type &operator*() & + { + return *reinterpret_cast(data().data()); + } +}; + +class C3ISPParams +{ +public: + C3ISPParams(Span data); + + template + C3ISPParamsBlock block() + { + return C3ISPParamsBlock(block(B)); + } + + size_t size() const { return used_; } + +private: + friend class C3ISPParamsBlockBase; + + Span block(BlockType type); + + Span data_; + size_t used_; + + std::map> blocks_; +}; + +} /* namespace ipa::c3isp */ + +} /* namespace libcamera */ From patchwork Thu Feb 27 10:57:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22888 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DC144BF415 for ; Thu, 27 Feb 2025 10:58:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A95B168753; Thu, 27 Feb 2025 11:58:52 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D17668749 for ; Thu, 27 Feb 2025 11:58:50 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:58:48 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 05/11] ipa: c3-isp: Add Agc algorithm Date: Thu, 27 Feb 2025 18:57:27 +0800 Message-ID: <20250227105733.187611-6-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Agc algorithm for the C3 ISP. The tuning files for uncalibrated sensors is extended to enable the algorithm. Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/agc.cpp | 310 ++++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/agc.h | 51 +++++ src/ipa/c3-isp/algorithms/meson.build | 2 +- src/ipa/c3-isp/data/meson.build | 5 +- src/ipa/c3-isp/data/uncalibrated.yaml | 7 + 5 files changed, 371 insertions(+), 4 deletions(-) create mode 100644 src/ipa/c3-isp/algorithms/agc.cpp create mode 100644 src/ipa/c3-isp/algorithms/agc.h create mode 100644 src/ipa/c3-isp/data/uncalibrated.yaml diff --git a/src/ipa/c3-isp/algorithms/agc.cpp b/src/ipa/c3-isp/algorithms/agc.cpp new file mode 100644 index 00000000..b9357cbe --- /dev/null +++ b/src/ipa/c3-isp/algorithms/agc.cpp @@ -0,0 +1,310 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include + +#include +#include + +#include +#include + +#include "libipa/histogram.h" + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::c3isp::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(C3ISPAgc) + +Agc::Agc() +{ +} + +/** + * \brief Initialise the AGC algorithm from tuning files + * \param[in] context The shared IPA context + * \param[in] tuningData The YamlObject containing Agc tuning data + * + * This function calls the base class' tuningData parsers to discover which + * control values are supported. + * + * \return 0 on success or errors from the base class + */ +int Agc::init(IPAContext &context, const YamlObject &tuningData) +{ + int ret; + + ret = parseTuningData(tuningData); + if (ret) + return ret; + + context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true); + + context.ctrlMap.merge(controls()); + + return 0; +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + const IPASessionConfiguration &configuration = context.configuration; + IPAActiveState &activeState = context.activeState; + + /* Configure the default gain and exposure */ + activeState.agc.autoEnabled = true; + activeState.agc.automatic.sensorGain = configuration.agc.minAnalogueGain; + activeState.agc.automatic.exposure = configuration.agc.defaultExposure; + activeState.agc.manual.sensorGain = configuration.agc.minAnalogueGain; + activeState.agc.manual.exposure = configuration.agc.defaultExposure; + activeState.agc.constraintMode = constraintModes().begin()->first; + activeState.agc.exposureMode = exposureModeHelpers().begin()->first; + + setLimits(configuration.agc.minShutterSpeed, + configuration.agc.maxShutterSpeed, + configuration.agc.minAnalogueGain, + configuration.agc.maxAnalogueGain); + + resetFrameCount(); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Agc::queueRequest(IPAContext &context, const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &constraintMode = controls.get(controls::AeConstraintMode); + agc.constraintMode = constraintMode.value_or(agc.constraintMode); + + const auto &exposureMode = controls.get(controls::AeExposureMode); + agc.exposureMode = exposureMode.value_or(agc.exposureMode); + + const auto &agcEnable = controls.get(controls::AeEnable); + if (agcEnable && *agcEnable != agc.autoEnabled) { + agc.autoEnabled = *agcEnable; + + LOG(C3ISPAgc, Info) + << (agc.autoEnabled ? "Enabling" : "Disablin") + << " AGC"; + } + + if (agc.autoEnabled) + return; + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure) { + agc.manual.exposure = *exposure * 1.0us / + context.configuration.sensor.lineDuration; + + LOG(C3ISPAgc, Debug) + << "Exposure set to " << agc.manual.exposure + << " on request sequence " << frame; + } + + const auto &analogueGain = controls.get(controls::AnalogueGain); + if (analogueGain) { + agc.manual.sensorGain = *analogueGain; + + LOG(C3ISPAgc, Debug) + << "Analogue gain set to " << agc.manual.sensorGain + << " on request sequence " << frame; + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Agc::prepare(IPAContext &context, const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + C3ISPParams *params) +{ + if (frame > 0) + return; + + auto AeCfg = params->block(); + AeCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + AeCfg->tap_point = C3_ISP_AE_STATS_TAP_MLS; + + /* A 17x15 grid */ + AeCfg->horiz_zones_num = 17; + AeCfg->vert_zones_num = 15; + + Span weights{ AeCfg->zone_weight, C3_ISP_AE_MAX_ZONES }; + std::fill(weights.begin(), weights.end(), 1); + + Size sensorSize = context.configuration.sensor.size; + uint8_t maxPointNum = + std::max(AeCfg->horiz_zones_num, AeCfg->vert_zones_num) + 1; + + for (unsigned int i = 0; i < maxPointNum; i++) { + uint16_t hidx = i * sensorSize.width / AeCfg->horiz_zones_num; + + /* Aligned with 2 */ + hidx = hidx / 2 * 2; + AeCfg->horiz_coord[i] = + std::min(hidx, (uint16_t)sensorSize.width); + + uint16_t vidx = i * sensorSize.height / AeCfg->vert_zones_num; + + /* Aligned with 2 */ + vidx = vidx / 2 * 2; + AeCfg->vert_coord[i] = + std::min(vidx, (uint16_t)sensorSize.height); + } +} + +Histogram Agc::parseStatistics(const c3_isp_stats_info *stats) +{ + const struct c3_isp_ae_stats *info = &stats->ae; + std::vector means(C3_ISP_AE_MAX_ZONES, 0); + + /* + * Each zone has a 5-bin histogram and the total sum is normalized to + * 65535. For the convenience of calculation, we use the centre of the + * 5-bin as target area. So it can be assumed that: + * hist1 represents the number of brightness 16, + * hist2 represents the number of brightness 72, + * hist3 represents the number of brightness 128, + * + * Finally, the average brightness of a zone can be calculated. + */ + for (unsigned int i = 0; i < C3_ISP_AE_MAX_ZONES; i++) { + uint16_t hist2 = 65535 - info->stats[i].hist0 - + info->stats[i].hist1 - + info->stats[i].hist3 - + info->stats[i].hist4; + + uint32_t lumaSum = info->stats[i].hist1 * 16 + + hist2 * 72 + + info->stats[i].hist3 * 128; + + uint32_t histSum = info->stats[i].hist1 + + hist2 + + info->stats[i].hist3; + + means[i] = lumaSum / histSum; + } + + lumaMeans_ = means; + + /* This is a 1024-bin histogram */ + uint32_t *hist = const_cast(info->hist); + + return Histogram(Span(hist, std::size(info->hist))); +} + +/** + * \brief Estimate the relative luminance of the frame with a given gain + * \param[in] gain The gain to apply in estimating luminance + * + * The estimation is based on the average value of the zone. Each + * average value is multiplied by the gain, and then saturated to + * to approximate the sensor behaviour at high brightness values. + * The approximation is quite rough, as it doesn't take into account + * non-linearities when approaching saturation. + * + * The values are normalized to the [0.0, 1.0] range, where 1.0 corresponds + * to a theoretical perfect reflector of 100% reference white. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Relative_luminance + * + * \return The relative luminance of the frame + */ +double Agc::estimateLuminance(double gain) const +{ + double sum = 0.0; + + for (unsigned int i = 0; i < lumaMeans_.size(); i++) + sum += std::min(lumaMeans_[i] * gain, 128.0); + + return sum / lumaMeans_.size() / 128; +} + +/** + * \brief Process C3 ISP statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number + * \param[in] frameContext The current frame context + * \param[in] stats The C3 ISP statistics and ISP results + * \param[out] metadata Metadata for the frame, to be filled by the algorithm + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const c3_isp_stats_info *stats, + ControlList &metadata) +{ + Histogram hist = parseStatistics(stats); + + utils::Duration shutterTime; + double aGain, dGain; + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + utils::Duration exposureTime = context.configuration.sensor.lineDuration * + frameContext.agc.exposure; + double analogueGain = frameContext.agc.sensorGain; + utils::Duration effectiveExposureValue = exposureTime * analogueGain; + + std::tie(shutterTime, aGain, dGain) = + calculateNewEv(context.activeState.agc.constraintMode, + context.activeState.agc.exposureMode, hist, + effectiveExposureValue); + + LOG(C3ISPAgc, Debug) + << "Shutter time, analogue gain and digital gain are " + << shutterTime << ", " << aGain << " and " << dGain; + + IPAActiveState &activeState = context.activeState; + + activeState.agc.automatic.exposure = shutterTime / context.configuration.sensor.lineDuration; + activeState.agc.automatic.sensorGain = aGain; + + metadata.set(controls::AnalogueGain, frameContext.agc.sensorGain); + metadata.set(controls::ExposureTime, exposureTime.get()); + + lumaMeans_ = {}; +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/agc.h b/src/ipa/c3-isp/algorithms/agc.h new file mode 100644 index 00000000..e7395903 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/agc.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include +#include + +#include + +#include "libipa/agc_mean_luminance.h" +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class Agc : public Algorithm, public AgcMeanLuminance +{ +public: + Agc(); + ~Agc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const c3_isp_stats_info *stats, + ControlList &metadata) override; +private: + Histogram parseStatistics(const c3_isp_stats_info *stats); + double estimateLuminance(double gain) const override; + + std::vector lumaMeans_; +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index 1e00af33..3e44fe91 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -1,5 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 c3isp_ipa_algorithms = files([ - + 'agc.cpp', ]) diff --git a/src/ipa/c3-isp/data/meson.build b/src/ipa/c3-isp/data/meson.build index 75e44bac..1e98cbcf 100644 --- a/src/ipa/c3-isp/data/meson.build +++ b/src/ipa/c3-isp/data/meson.build @@ -1,9 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ - + 'uncalibrated.yaml' ]) install_data(conf_files, - install_dir : ipa_data_dir / 'c3isp', - install_tag : 'runtime') + install_dir : ipa_data_dir / 'c3isp') diff --git a/src/ipa/c3-isp/data/uncalibrated.yaml b/src/ipa/c3-isp/data/uncalibrated.yaml new file mode 100644 index 00000000..6dcc0295 --- /dev/null +++ b/src/ipa/c3-isp/data/uncalibrated.yaml @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Agc: +... From patchwork Thu Feb 27 10:57:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22889 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 69ED1BF415 for ; Thu, 27 Feb 2025 10:58:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 277FB6875D; Thu, 27 Feb 2025 11:58:56 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F394A68749 for ; Thu, 27 Feb 2025 11:58:54 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:58:53 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 06/11] ipa: c3-isp: Add Awb algorithm Date: Thu, 27 Feb 2025 18:57:28 +0800 Message-ID: <20250227105733.187611-7-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Awb algorithm for the C3 ISP. Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/awb.cpp | 200 ++++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/awb.h | 39 +++++ src/ipa/c3-isp/algorithms/meson.build | 1 + 3 files changed, 240 insertions(+) create mode 100644 src/ipa/c3-isp/algorithms/awb.cpp create mode 100644 src/ipa/c3-isp/algorithms/awb.h diff --git a/src/ipa/c3-isp/algorithms/awb.cpp b/src/ipa/c3-isp/algorithms/awb.cpp new file mode 100644 index 00000000..786dcc81 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/awb.cpp @@ -0,0 +1,200 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP AWB control algorithm + */ + +#include "awb.h" + +#include + +#include +#include + +#include + +#include "libipa/colours.h" +#include "libipa/fixedpoint.h" + +/** + * \file awb.h + */ + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +/** + * \class Awb + * \brief A Grey world white balance correction algorithm + */ + +LOG_DEFINE_CATEGORY(C3ISPAwb) + +Awb::Awb() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Awb::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + context.activeState.awb.rGain = 1.0; + context.activeState.awb.bGain = 1.0; + + return 0; +} + +void Awb::fillGainsParam(IPAContext &context, IPAFrameContext &frameContext, + C3ISPParams *params) +{ + double rGain = context.activeState.awb.rGain; + double bGain = context.activeState.awb.bGain; + + auto AWBGains = params->block(); + AWBGains.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + AWBGains->gr_gain = floatingToFixedPoint<4, 8, uint16_t, double>(1.0); + AWBGains->r_gain = floatingToFixedPoint<4, 8, uint16_t, double>(rGain); + AWBGains->b_gain = floatingToFixedPoint<4, 8, uint16_t, double>(bGain); + AWBGains->gb_gain = floatingToFixedPoint<4, 8, uint16_t, double>(1.0); + + frameContext.awb.rGain = rGain; + frameContext.awb.bGain = bGain; +} + +void Awb::fillConfigParam(IPAContext &context, C3ISPParams *params) +{ + auto AWBCfg = params->block(); + AWBCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + AWBCfg->tap_point = C3_ISP_AWB_STATS_TAP_BEFORE_WB; + AWBCfg->satur_vald = 1; + + /* We use the full 32x24 zoning scheme */ + AWBCfg->horiz_zones_num = 32; + AWBCfg->vert_zones_num = 24; + + /* The ratios themselves are stored in Q4.8 format */ + AWBCfg->rg_min = 75; + AWBCfg->rg_max = 256; + AWBCfg->bg_min = 44; + AWBCfg->bg_max = 222; + + /* The triming of ratios are stored in Q4.8 format */ + AWBCfg->rg_low = 93; + AWBCfg->rg_high = 244; + AWBCfg->bg_low = 61; + AWBCfg->bg_high = 205; + + Span weights{ AWBCfg->zone_weight, C3_ISP_AWB_MAX_ZONES }; + std::fill(weights.begin(), weights.end(), 1); + + Size sensorSize = context.configuration.sensor.size; + uint8_t maxPointNum = + std::max(AWBCfg->horiz_zones_num, AWBCfg->vert_zones_num) + 1; + + for (unsigned int i = 0; i < maxPointNum; i++) { + uint16_t hidx = i * sensorSize.width / AWBCfg->horiz_zones_num; + + /* Aligned with 2 */ + hidx = hidx / 2 * 2; + AWBCfg->horiz_coord[i] = std::min(hidx, (uint16_t)sensorSize.width); + + uint16_t vidx = i * sensorSize.height / AWBCfg->vert_zones_num; + + /* Aligned with 2 */ + vidx = vidx / 2 * 2; + AWBCfg->vert_coord[i] = std::min(vidx, (uint16_t)sensorSize.height); + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, C3ISPParams *params) +{ + fillGainsParam(context, frameContext, params); + + if (frame > 0) + return; + + fillConfigParam(context, params); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const c3_isp_stats_info *stats, + ControlList &metadata) +{ + IPAActiveState &activeState = context.activeState; + const struct c3_isp_awb_stats *awb = &stats->awb; + + /* The AWB statistics are R:G and B:G ratios of the zones. */ + uint32_t counted_zones = 0; + double rgSum = 0, bgSum = 0; + + for (unsigned int i = 0; i < C3_ISP_AWB_MAX_ZONES; i++) { + if (!awb->stats[i].pixel_sum) + continue; + + /* The statistics are in Q4.12 format. */ + rgSum += fixedToFloatingPoint<4, 12, double, uint16_t>(awb->stats[i].rg); + bgSum += fixedToFloatingPoint<4, 12, double, uint16_t>(awb->stats[i].bg); + + counted_zones++; + } + + double rgAvg, bgAvg; + if (!counted_zones) { + rgAvg = 1.0; + bgAvg = 1.0; + } else { + rgAvg = rgSum / counted_zones; + bgAvg = bgSum / counted_zones; + } + + /* + * To simplify the calculation, the average green is hardcoded + * to 1.0. + */ + activeState.awb.temperatureK = estimateCCT({ { rgAvg, 1.0, bgAvg } }); + + /* Metadata shall contain the up to date measurement */ + metadata.set(controls::ColourTemperature, activeState.awb.temperatureK); + + /* + * Estimate the red and blue gains to apply in a grey world. + * The green gain is hardcoded to 1.0. Avoid division by zero + * by clamping the divisor to mininum value of 0.0625. + */ + double rGain = 1 / std::max(rgAvg, 0.0625); + double bGain = 1 / std::max(bgAvg, 0.0625); + + /* Filter the values to avoid oscillations. */ + double speed = 0.2; + rGain = speed * rGain + (1 - speed) * activeState.awb.rGain; + bGain = speed * bGain + (1 - speed) * activeState.awb.bGain; + + activeState.awb.rGain = rGain; + activeState.awb.bGain = bGain; + + metadata.set(controls::ColourGains, { static_cast(frameContext.awb.rGain), + static_cast(frameContext.awb.bGain) }); + + LOG(C3ISPAwb, Debug) << "Gains: R " << activeState.awb.rGain + << ", B " << activeState.awb.bGain + << ", Ct " << activeState.awb.temperatureK << "K"; +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/awb.h b/src/ipa/c3-isp/algorithms/awb.h new file mode 100644 index 00000000..bd036c11 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/awb.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP AWB control algorithm + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class Awb : public Algorithm +{ +public: + Awb(); + ~Awb() = default; + + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const c3_isp_stats_info *stats, + ControlList &metadata) override; + +private: + void fillGainsParam(IPAContext &context, IPAFrameContext &frameContext, + C3ISPParams *params); + void fillConfigParam(IPAContext &context, C3ISPParams *params); +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index 3e44fe91..9460da99 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -2,4 +2,5 @@ c3isp_ipa_algorithms = files([ 'agc.cpp', + 'awb.cpp', ]) From patchwork Thu Feb 27 10:57:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22890 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E5AFBBF415 for ; Thu, 27 Feb 2025 10:58:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 80A5968767; Thu, 27 Feb 2025 11:58:59 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9009368751 for ; Thu, 27 Feb 2025 11:58:55 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:58:54 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 07/11] ipa: c3-isp: Add BLC Algorithm Date: Thu, 27 Feb 2025 18:57:29 +0800 Message-ID: <20250227105733.187611-8-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Black Level Correction algorithm. Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/blc.cpp | 151 ++++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/blc.h | 45 ++++++++ src/ipa/c3-isp/algorithms/meson.build | 1 + 3 files changed, 197 insertions(+) create mode 100644 src/ipa/c3-isp/algorithms/blc.cpp create mode 100644 src/ipa/c3-isp/algorithms/blc.h diff --git a/src/ipa/c3-isp/algorithms/blc.cpp b/src/ipa/c3-isp/algorithms/blc.cpp new file mode 100644 index 00000000..4410d03f --- /dev/null +++ b/src/ipa/c3-isp/algorithms/blc.cpp @@ -0,0 +1,151 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Black Level Correction control + */ + +#include "blc.h" + +#include + +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +/** + * \file blc.h + */ + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +/** + * \class Blc + * \brief C3 ISP Black Level Correction control + * + * The pixels output by the camera normally include a black level, because + * sensors do not always report a signal level of '0' for black. Pixels at or + * below this level should be considered black. To achieve that, the C3 ISP BLC + * algorithm subtracts a configurable offset from all pixels. + * + * The black level can be measured at runtime from an optical dark region of the + * camera sensor, or measured during the camera tuning process. The first option + * isn't currently supported. + */ + +LOG_DEFINE_CATEGORY(C3ISPBlc) + +Blc::Blc() + : tuningBlc_(false) +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Blc::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + offsetR_ = tuningData["offsetR"].get(0); + offsetGr_ = tuningData["offsetGr"].get(0); + offsetGb_ = tuningData["offsetGb"].get(0); + offsetB_ = tuningData["offsetB"].get(0); + + if (!(offsetR_ + offsetGr_ + offsetGb_ + offsetB_)) { + LOG(C3ISPBlc, Debug) << "All black level offsets are empty"; + tuningBlc_ = false; + } else { + tuningBlc_ = true; + } + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Blc::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + uint16_t blackLevel = offsetCompress(context.configuration.sensor.blackLevel); + + if (blackLevel && !tuningBlc_) { + offsetR_ = context.configuration.sensor.blackLevel; + offsetGr_ = context.configuration.sensor.blackLevel; + offsetGb_ = context.configuration.sensor.blackLevel; + offsetB_ = context.configuration.sensor.blackLevel; + } + + return 0; +} + +uint16_t Blc::offsetCompress(uint32_t blackLevel) +{ + uint8_t shift = 0; + uint32_t integer = 0; + uint8_t highestBitPos = static_cast(log2(blackLevel)); + + /* The compressed offset is stored in Q4.12 format */ + if (highestBitPos < 12) { + shift = 0; + integer = blackLevel; + } else { + shift = highestBitPos - 11; + integer = blackLevel >> shift; + } + + shift = (shift > 0xf) ? 0xf : shift; + + return (shift & 0xf) << 12 | (integer & 0xfff); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Blc::prepare([[maybe_unused]] IPAContext &context, const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + C3ISPParams *params) +{ + if (frame > 0) + return; + + if (!tuningBlc_ && !context.configuration.sensor.blackLevel) + return; + + auto blcCfg = params->block(); + blcCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + blcCfg->gr_ofst = offsetCompress(offsetGr_); + blcCfg->r_ofst = offsetCompress(offsetR_); + blcCfg->b_ofst = offsetCompress(offsetB_); + blcCfg->gb_ofst = offsetCompress(offsetGb_); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Blc::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const c3_isp_stats_info *stats, + ControlList &metadata) +{ + /* + * Black Level Offsets in tuning data need to be 20-bit, whereas the + * metadata expects values from a 16-bit range. Right-shift to remove + * the 4 least significant bits. + */ + metadata.set(controls::SensorBlackLevels, + { static_cast(offsetR_ >> 4), + static_cast(offsetGr_ >> 4), + static_cast(offsetGb_ >> 4), + static_cast(offsetB_ >> 4) }); +} + +REGISTER_IPA_ALGORITHM(Blc, "Blc") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/blc.h b/src/ipa/c3-isp/algorithms/blc.h new file mode 100644 index 00000000..67244e4a --- /dev/null +++ b/src/ipa/c3-isp/algorithms/blc.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Black Level Correction control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class Blc : public Algorithm +{ +public: + Blc(); + ~Blc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const c3_isp_stats_info *stats, + ControlList &metadata) override; + +private: + uint16_t offsetCompress(uint32_t blackLevel); + + bool tuningBlc_; + uint32_t offsetR_; + uint32_t offsetGr_; + uint32_t offsetGb_; + uint32_t offsetB_; +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index 9460da99..d91973e3 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -3,4 +3,5 @@ c3isp_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', + 'blc.cpp', ]) From patchwork Thu Feb 27 10:57:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22891 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76ADDBF415 for ; Thu, 27 Feb 2025 10:59:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 10F9B68766; Thu, 27 Feb 2025 11:59:01 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DC5168760 for ; Thu, 27 Feb 2025 11:58:56 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:58:55 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 08/11] ipa: c3-isp: Add CCM Algorithm Date: Thu, 27 Feb 2025 18:57:30 +0800 Message-ID: <20250227105733.187611-9-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Colour Correction Matrix algorithm Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/ccm.cpp | 92 +++++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/ccm.h | 36 +++++++++++ src/ipa/c3-isp/algorithms/meson.build | 1 + 3 files changed, 129 insertions(+) create mode 100644 src/ipa/c3-isp/algorithms/ccm.cpp create mode 100644 src/ipa/c3-isp/algorithms/ccm.h diff --git a/src/ipa/c3-isp/algorithms/ccm.cpp b/src/ipa/c3-isp/algorithms/ccm.cpp new file mode 100644 index 00000000..c0efb539 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/ccm.cpp @@ -0,0 +1,92 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Color Correction Matrix control + */ + +#include "ccm.h" + +#include +#include + +#include + +#include "libcamera/internal/yaml_parser.h" + +/** + * \file ccm.h + */ + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +/** + * \class Ccm + * \brief A color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(C3ISPCcm) + +Ccm::Ccm() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + ccmCoeff_ = tuningData["ccmCoeff"].getList().value_or(std::vector{}); + if (ccmCoeff_.size() != 9) { + LOG(C3ISPCcm, Error) << "Invalid CCM coeff size"; + return -EINVAL; + } + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, C3ISPParams *params) +{ + if (frame > 0) + return; + + auto CcmCfg = params->block(); + CcmCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) { + CcmCfg->matrix[i][j] = ccmCoeff_[i * 3 + j]; + } + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const c3_isp_stats_info *stats, + ControlList &metadata) +{ + float m[9]; + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + m[i * 3 + j] = ccmCoeff_[i * 3 + j] / 256.0; + } + + metadata.set(controls::ColourCorrectionMatrix, m); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/ccm.h b/src/ipa/c3-isp/algorithms/ccm.h new file mode 100644 index 00000000..c65e7edf --- /dev/null +++ b/src/ipa/c3-isp/algorithms/ccm.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Color Correction Matrix control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm(); + ~Ccm() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const c3_isp_stats_info *stats, + ControlList &metadata) override; + +private: + std::vector ccmCoeff_; +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index d91973e3..591c2433 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -4,4 +4,5 @@ c3isp_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'ccm.cpp', ]) From patchwork Thu Feb 27 10:57:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22892 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E905C3264 for ; Thu, 27 Feb 2025 10:59:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D03468763; Thu, 27 Feb 2025 11:59:02 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F218A68763 for ; Thu, 27 Feb 2025 11:58:56 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 18:58:56 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 09/11] ipa: c3-isp: Add CSC Algorithm Date: Thu, 27 Feb 2025 18:57:31 +0800 Message-ID: <20250227105733.187611-10-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Colour Space Coversion algorithm. Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/csc.cpp | 66 +++++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/csc.h | 32 +++++++++++++ src/ipa/c3-isp/algorithms/meson.build | 1 + 3 files changed, 99 insertions(+) create mode 100644 src/ipa/c3-isp/algorithms/csc.cpp create mode 100644 src/ipa/c3-isp/algorithms/csc.h diff --git a/src/ipa/c3-isp/algorithms/csc.cpp b/src/ipa/c3-isp/algorithms/csc.cpp new file mode 100644 index 00000000..588b1e19 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/csc.cpp @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Color Space Conversion control + */ + +#include "csc.h" + +#include + +/** + * \file csc.h + */ + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +/** + * \class Csc + * \brief Color Space Conversion algorithm + */ + +LOG_DEFINE_CATEGORY(C3ISPCsc) + +Csc::Csc() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Csc::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + cscCoeff_ = tuningData["cscCoeff"].getList().value_or(std::vector{}); + if (cscCoeff_.size() != 9) { + LOG(C3ISPCsc, Error) << "Invalid CSC coeff size"; + return -EINVAL; + } + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Csc::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + C3ISPParams *params) +{ + auto CscCfg = params->block(); + CscCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + CscCfg->matrix[i][j] = cscCoeff_[i * 3 + j]; + } +} + +REGISTER_IPA_ALGORITHM(Csc, "Csc") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/csc.h b/src/ipa/c3-isp/algorithms/csc.h new file mode 100644 index 00000000..ec082a41 --- /dev/null +++ b/src/ipa/c3-isp/algorithms/csc.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Color Space Conversion control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class Csc : public Algorithm +{ +public: + Csc(); + ~Csc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; +private: + std::vector cscCoeff_; +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index 591c2433..5e7e56db 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -5,4 +5,5 @@ c3isp_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'ccm.cpp', + 'csc.cpp', ]) From patchwork Thu Feb 27 10:57:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22893 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7C67BF415 for ; Thu, 27 Feb 2025 11:00:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 418E168757; Thu, 27 Feb 2025 12:00:14 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 259BC60322 for ; Thu, 27 Feb 2025 12:00:13 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 19:00:11 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 10/11] ipa: c3-isp: Add Post Gamma Algorithm Date: Thu, 27 Feb 2025 18:57:32 +0800 Message-ID: <20250227105733.187611-11-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Post Gamma algorithm. Signed-off-by: Keke Li --- src/ipa/c3-isp/algorithms/meson.build | 1 + src/ipa/c3-isp/algorithms/post_gamma.cpp | 65 ++++++++++++++++++++++++ src/ipa/c3-isp/algorithms/post_gamma.h | 33 ++++++++++++ 3 files changed, 99 insertions(+) create mode 100644 src/ipa/c3-isp/algorithms/post_gamma.cpp create mode 100644 src/ipa/c3-isp/algorithms/post_gamma.h diff --git a/src/ipa/c3-isp/algorithms/meson.build b/src/ipa/c3-isp/algorithms/meson.build index 5e7e56db..57600f12 100644 --- a/src/ipa/c3-isp/algorithms/meson.build +++ b/src/ipa/c3-isp/algorithms/meson.build @@ -6,4 +6,5 @@ c3isp_ipa_algorithms = files([ 'blc.cpp', 'ccm.cpp', 'csc.cpp', + 'post_gamma.cpp', ]) diff --git a/src/ipa/c3-isp/algorithms/post_gamma.cpp b/src/ipa/c3-isp/algorithms/post_gamma.cpp new file mode 100644 index 00000000..f8c1a1da --- /dev/null +++ b/src/ipa/c3-isp/algorithms/post_gamma.cpp @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Post Gamma control + */ + +#include "post_gamma.h" + +#include + +/** + * \file post_gamma.h + */ + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +/** + * \class PostGamma + * \brief A post gamma algorithm + */ + +LOG_DEFINE_CATEGORY(C3ISPPostGamma) + +PostGamma::PostGamma() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int PostGamma::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + gammaLut_ = tuningData["gammaLut"].getList().value_or(std::vector{}); + if (gammaLut_.size() != 129) { + LOG(C3ISPPostGamma, Error) << "Invalid gamma Look-up table size"; + return -EINVAL; + } + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void PostGamma::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + C3ISPParams *params) +{ + auto PostGammaCfg = params->block(); + PostGammaCfg.setEnabled(C3_ISP_PARAMS_BLOCK_FL_ENABLE); + + for (unsigned int i = 0; i < 129; i++) { + PostGammaCfg->lut[i] = gammaLut_[i]; + } +} + +REGISTER_IPA_ALGORITHM(PostGamma, "PostGamma") + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/c3-isp/algorithms/post_gamma.h b/src/ipa/c3-isp/algorithms/post_gamma.h new file mode 100644 index 00000000..44d8f9db --- /dev/null +++ b/src/ipa/c3-isp/algorithms/post_gamma.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Amlogic + * + * C3ISP Post Gamma control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::c3isp::algorithms { + +class PostGamma : public Algorithm +{ +public: + PostGamma(); + ~PostGamma() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + C3ISPParams *params) override; + +private: + std::vector gammaLut_; +}; + +} /* namespace ipa::c3isp::algorithms */ + +} /* namespace libcamera */ From patchwork Thu Feb 27 10:57:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keke Li X-Patchwork-Id: 22894 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9266BBF415 for ; Thu, 27 Feb 2025 11:00:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 606CA6875D; Thu, 27 Feb 2025 12:00:16 +0100 (CET) Received: from mail-sh.amlogic.com (unknown [58.32.228.46]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C9BBE68718 for ; Thu, 27 Feb 2025 12:00:13 +0100 (CET) Received: from droid10.amlogic.com (10.18.11.213) by mail-sh.amlogic.com (10.18.11.5) with Microsoft SMTP Server id 15.1.2507.39; Thu, 27 Feb 2025 19:00:12 +0800 From: Keke Li To: CC: , , , Keke Li Subject: [PATCH v3 11/11] ipa: c3-isp: Add IMX290 tuning data file Date: Thu, 27 Feb 2025 18:57:33 +0800 Message-ID: <20250227105733.187611-12-keke.li@amlogic.com> X-Mailer: git-send-email 2.29.0 In-Reply-To: <20250227105733.187611-1-keke.li@amlogic.com> References: <20250227105733.187611-1-keke.li@amlogic.com> MIME-Version: 1.0 X-Originating-IP: [10.18.11.213] X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning data file for the IMX290 camera sensor. The balck level offset data is drawn from the camera sensor's datasheet. Signed-off-by: Keke Li --- src/ipa/c3-isp/data/imx290.yaml | 30 ++++++++++++++++++++++++++++++ src/ipa/c3-isp/data/meson.build | 1 + 2 files changed, 31 insertions(+) create mode 100644 src/ipa/c3-isp/data/imx290.yaml diff --git a/src/ipa/c3-isp/data/imx290.yaml b/src/ipa/c3-isp/data/imx290.yaml new file mode 100644 index 00000000..a3de98b0 --- /dev/null +++ b/src/ipa/c3-isp/data/imx290.yaml @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Agc: + - Awb: + - PostGamma: + gammaLut: [ + 0, 86, 134, 169, 198, 223, 245, 265, 283, 300, 316, 331, + 346, 359, 373, 385, 397, 409, 420, 431, 441, 452, 462, 471, + 481, 490, 499, 508, 516, 525, 533, 541, 549, 557, 565, 572, + 580, 587, 594, 601, 608, 615, 622, 629, 635, 642, 648, 655, + 661, 667, 673, 679, 685, 691, 697, 703, 708, 714, 720, 725, + 731, 736, 742, 747, 752, 758, 763, 768, 773, 778, 783, 788, + 793, 798, 803, 808, 812, 817, 822, 827, 831, 836, 840, 845, + 849, 854, 858, 863, 867, 872, 876, 880, 884, 889, 893, 897, + 901, 905, 910, 914, 918, 922, 926, 930, 934, 938, 942, 946, + 950, 953, 957, 961, 965, 969, 972, 976, 980, 984, 987, 991, + 995, 998, 1002, 1006, 1009, 1013, 1016, 1020, 1023 + ] + - Ccm: + ccmCoeff: [ 533, -191, -86, -147, 474, -71, 23, -208, 441 ] + - Csc: + cscCoeff: [ 54, 183, 19, -29, -99, 128, 128, -116, -12 ] + - Blc: + offsetR: 61440 + offsetGr: 61440 + offsetGb: 61440 + offsetB: 61440 diff --git a/src/ipa/c3-isp/data/meson.build b/src/ipa/c3-isp/data/meson.build index 1e98cbcf..d87e4c80 100644 --- a/src/ipa/c3-isp/data/meson.build +++ b/src/ipa/c3-isp/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'imx290.yaml', 'uncalibrated.yaml' ])