From patchwork Fri Dec 10 11:21:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15112 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 86419C324B for ; Fri, 10 Dec 2021 11:22:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C4BF608E8; Fri, 10 Dec 2021 12:22:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kP9r44re"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4CC2D6088E for ; Fri, 10 Dec 2021 12:22:09 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id y196so6545193wmc.3 for ; Fri, 10 Dec 2021 03:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iw1oXJ4y6qHk5Vh9ACkUHIhW6uieykJz0tu6MkgxSbg=; b=kP9r44rexQ2r71m2z9pSpOWMlBQ/SpakNxn9udNZR8c+RpbGW8U5MiJ+sFnJvjCBka OkljS+XtSAPjLkz0ZXcGCu+/pXvO/mewzlJ+rIWpTVOdJxuXK7QCCzpV5jWW9FUHdc1Z aUNOShaZ4ikyL/Ovdl4YR5FLPJW10t/r2Yu/mfY6XfgsgsshHsQj9BPC4sWefQDxG2xq 1dNGuxi5fKq1IZrE3vZoDzgmQveTFIaEkCHdaJTgqMbeuO2peHgehmsHSeIwwoF3QfyI 6z358j1DWtnH0wcLBvX0w0vKzauhfM8Khc773+lj8cmiSL76UaZI5Iya4zOMrmZxziDp wgXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iw1oXJ4y6qHk5Vh9ACkUHIhW6uieykJz0tu6MkgxSbg=; b=u5mi7RZLKH2PXpUyjPWRZwD4jxCpVQtSpoiIhIzm4YL7bLMWIq5ZkWyjGCF6K3HZ7I fxEni+G4cH8d5nh2AWBdt6Vcb099/gFGiRVqEm5deDUXVnZ/0Y6I5Jx2CHOmt6AhBTA1 MIkGZoYDSN3BzfCaBxiFM69w/lly3l8ZcZXQeL4LYL3YPIBTh8/SVeMPJN9AAAyii6zu vQImStKCITrV6UVqGAFUduY15+98LagY1WF1kuR0p0n3sYkuM2eCkq/0vynZcBiU+cXm V9wafu49NULRydqGrfQuDOm8sY91EdrMkNn6ntaDuMEchPAVTPanJZ6WaKdPPVKOhtSO XJyw== X-Gm-Message-State: AOAM531UUJOrk4ZlSHpfrAKxPMsR5D5nLNqk7rBeEb190s/7rf9V7gjr FlkXe0oUvOJ3dZQ4gSiuXNeLO80mSc+0+A== X-Google-Smtp-Source: ABdhPJyGCaGpfkL82qLEopPdhkhwmi4enN0raSRbBX+PXkXdOhmjzx4x2XfaQ1DaR6I5f335mDvnyA== X-Received: by 2002:a7b:ce8c:: with SMTP id q12mr16007978wmj.91.1639135328970; Fri, 10 Dec 2021 03:22:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:08 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:42 +0000 Message-Id: <20211210112142.18441-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 86851ac4..eb74d96c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,6 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -132,6 +133,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -329,6 +331,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -496,11 +500,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Debug) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -524,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +558,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +566,12 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + /* + * Still image codecs usually expect the JPEG color space. + * Even RGB codecs will be fine as the RGB we get with the + * JPEG color space is the same as sRGB. + */ + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = sensorSize; bufferCount = 1; @@ -563,6 +589,11 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* + * Choose a color space appropriate for video recording. + * Rec.709 will be a good default for HD resolutions. + */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +602,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -617,6 +649,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -724,6 +757,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -739,6 +773,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -763,6 +801,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)