From patchwork Thu Nov 4 13:58:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14464 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9C6FC324E for ; Thu, 4 Nov 2021 13:58:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6292660352; Thu, 4 Nov 2021 14:58:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NRF75VlA"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4AFC36034F for ; Thu, 4 Nov 2021 14:58:27 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id 67-20020a1c1946000000b0030d4c90fa87so4355036wmz.2 for ; Thu, 04 Nov 2021 06:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=NRF75VlAaoeHCI5komFAHp4swHbQny90tLbCh4pl+GDMbwQS1rc3MMrTVWypiul4VN fHD2+x53Jk4CaqoelioERZYNcakJpk1hOjJsK4TIxtOlH6wz0kE4hdlWnQqNs9hZip3U t7bKZf2WvZGVHAoKi0EO9xxzMdvADCNal2iKg0CCiCsbrXG4oaEmrUkseMX3evCi0/6V p0BIacveiIRCMr7BDmwBQ9h8AyeiNO40R/EbO+zB4Gk+caHGg/YlcyAbGg1vT7Wl/LBR bSs6pv72L3/6DE3qV2E0ODDiwM1y+LSVLFyKLLuB2ik8r3RzSNQ/wFb1vO/cfvf34z48 NiKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=AfOKoM0Xxd+V/f1b2rIZJMchzZhwndB4h9JLTpiGJs5vcgcgsOX8yeCkFGHFqJiQ0a ddpn7RdUca2z559jSQz4bvOKXlFw2UL6CTIt4AF+JD+IgdtDn0dP3f8ZJ4fPBXs+PvSa B6xPctzzleIw6h08sE4dMfkvpFBfK1oH9HIvXIVUsJMxSZyXPc/YIMXnVef/XAYwGb9R EbRuSyF8qPK+Tz8N16shR2MCLiTiHXmfjLhd74bMTZ4/BAl7LpqrEb1GegzK7zNFYO+a Ka6Wxc4V6vn1bzPhCyztg5ChHmm/nJX6y5PjHBCdks6KCrCLZRhlXuMxPKKQjZu+Z77m ItXA== X-Gm-Message-State: AOAM532N5Vb+ciXssYBPWOKHC99i1H/owzZhlJw9hDiIc+K/Q1oZBtZ8 FCktBPXOFbXCNUVHyL+90U3fz16kNz+2Ug== X-Google-Smtp-Source: ABdhPJy+rQao/v+ex3rNuP/WSTLIB7o4Q+Ann03LkpPS8XGYLgdYEkZRhW+znZP6UR5m8h8y70KQ1A== X-Received: by 2002:a05:600c:210d:: with SMTP id u13mr23082571wml.82.1636034306844; Thu, 04 Nov 2021 06:58:26 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:26 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:05 +0000 Message-Id: <20211104135805.5269-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..bbb21e9b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.requestedColorSpace; + LOG(RPI, Debug) + << "Try color space " << cfg.requestedColorSpace.toString(); int ret = dev->tryFormat(&format); if (ret) return Invalid; + cfg.actualColorSpace = format.colorSpace; + if (cfg.actualColorSpace != cfg.requestedColorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << cfg.requestedColorSpace.toString() << " to " + << cfg.actualColorSpace.toString(); + } cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + ColorSpace colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.requestedColorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.requestedColorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (cfg.actualColorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space change (" + << cfg.actualColorSpace.toString() << " to " + << format.colorSpace.toString() << ") in stream " + << stream->name(); + cfg.actualColorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << cfg.actualColorSpace.toString(); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)