From patchwork Thu Dec 9 10:12:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15098 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D484C3258 for ; Thu, 9 Dec 2021 10:13:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ABBC06088E; Thu, 9 Dec 2021 11:13:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rWvjyPcj"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3B18B60894 for ; Thu, 9 Dec 2021 11:13:37 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id o19-20020a1c7513000000b0033a93202467so3661656wmc.2 for ; Thu, 09 Dec 2021 02:13:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=smk/dJpPZW6o/pfsEEykw+VPU4y38gdL3mRud2K8yx4=; b=rWvjyPcjSWtJUF9jy64Le/Ri4pr52efNSxx5CMuS7kBlYo7w0UA6DCEJ7o6F4oTucG oyhMq2UVoQCumXWVZAkfcsH26MZY8ry6byplKorW3FFOdyjLwUgIb/ehDTsKNx/PQ4ec OWkBWzQZSxBkrTWqreJrx9+m7gbGHLKOk+qW6cjaPmGvdQZC+V7gNMH8kFVM7FrZFE12 ACwP51KUqIGt6X27P9hSkKrHawVznxe7U1hc3W6ZzDrOvdu+na2Kidr2REa/8RdEz5So fHMYmtDIeYshIZ0QxGcO6uSvAYWiS+KUWyHvBnJfUbkUB3BLlrlBdv8HsUZEdsx7KtK7 Q2pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=smk/dJpPZW6o/pfsEEykw+VPU4y38gdL3mRud2K8yx4=; b=GwKhkSdQXoceBgRVMDyYTc5Codxbd2FRAg3cilfvO+n3vKvZYxBYappbezBJ0h6SsR XH8oBVNfcHF3Qtj1M70QwQGYRe6x/hzRbUQ5OpuMW8gAL6suO7EFP7AMlyBXGk+GF4xO wNvNg6jbVPdB2H/rCb0ptntsRuWEs11YjreL4nlXOoMqOuC3tgySwZu8unqqiyjioFrH 0Eo1XNpWcFZA9o7Tv7eYkpW2RR/L4+ZUyuuPTEQ4tgyFD25c88qH4UZfBC6VbLllr+Uo KMU18oNhcvOo2xxILUUG3FSKJ8l1hWKMbLk1mEfpZGAV46Mgw3j7BB4EhWK7q+dIHVZD Rcjw== X-Gm-Message-State: AOAM533b2X9NNyCKIEE8iwVFCU4Dg9Cgnpj46A78mlKFvtgEPwoIf0yb kjGniVsM9ZO2Rw8S5lJeJ1ijX9uM/2l2Dq8P X-Google-Smtp-Source: ABdhPJxFJa5owrHqje5gRuKNp9osGu80OVHDa8+JzYJMHPXdULAVqH4Li3gB2zZDdCaFOWwqBa4M8g== X-Received: by 2002:a05:600c:2c4a:: with SMTP id r10mr6096688wmg.125.1639044816653; Thu, 09 Dec 2021 02:13:36 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:36 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:45 +0000 Message-Id: <20211209101245.6187-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 22c8ee68..368953e1 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,6 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -132,6 +133,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -329,6 +331,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -496,11 +500,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -524,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -538,6 +557,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -545,6 +565,12 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + /* + * Still image codecs usually expect the JPEG color space. + * Even RGB codecs will be fine as the RGB we get with the + * JPEG color space is the same as sRGB. + */ + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -562,6 +588,11 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* + * Choose a color space appropriate for video recording. + * Rec.709 will be a good default for HD resolutions. + */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -570,6 +601,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -612,6 +644,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -719,6 +752,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -734,6 +768,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -758,6 +796,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)