{"id":8925,"url":"https://patchwork.libcamera.org/api/patches/8925/?format=json","web_url":"https://patchwork.libcamera.org/patch/8925/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200722134303.GA27212@kaaira-HP-Pavilion-Notebook>","date":"2020-07-22T13:43:04","name":"[libcamera-devel] media: vimc: VIMC multiple streams","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"65fbc98e54c7af4b280a159657fb45f3ac1c07ff","submitter":{"id":39,"url":"https://patchwork.libcamera.org/api/people/39/?format=json","name":"Kaaira Gupta","email":"kgupta@es.iitr.ac.in"},"delegate":{"id":11,"url":"https://patchwork.libcamera.org/api/users/11/?format=json","username":"kbingham","first_name":"Kieran","last_name":"Bingham","email":"kieran.bingham@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/8925/mbox/","series":[{"id":1128,"url":"https://patchwork.libcamera.org/api/series/1128/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1128","date":"2020-07-22T13:43:04","name":"[libcamera-devel] media: vimc: VIMC multiple streams","version":1,"mbox":"https://patchwork.libcamera.org/series/1128/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/8925/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/8925/checks/","tags":{},"headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B3B21BDB1B\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 22 Jul 2020 13:43:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 472A660948;\n\tWed, 22 Jul 2020 15:43:12 +0200 (CEST)","from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com\n\t[IPv6:2607:f8b0:4864:20::1041])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 798FB6053C\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 22 Jul 2020 15:43:11 +0200 (CEST)","by mail-pj1-x1041.google.com with SMTP id k1so1498905pjt.5\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 22 Jul 2020 06:43:11 -0700 (PDT)","from kaaira-HP-Pavilion-Notebook ([103.113.213.178])\n\tby smtp.gmail.com with ESMTPSA id\n\tl16sm23941473pff.167.2020.07.22.06.43.08\n\t(version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256);\n\tWed, 22 Jul 2020 06:43:09 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=es-iitr-ac-in.20150623.gappssmtp.com\n\theader.i=@es-iitr-ac-in.20150623.gappssmtp.com\n\theader.b=\"pcu/eiL5\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=es-iitr-ac-in.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:subject:message-id:mime-version:content-disposition\n\t:user-agent; bh=y4lW4sfL0CUAZ4TNRnAc1oO9jlALN1Z/xI/HJ0V+T2k=;\n\tb=pcu/eiL59R65OaYMVLlo3XgkMt01Z4xGSPnox3zmJQSoNCFMgdtDlrEX6M1Xaduz1a\n\tO0s1awVw63FYCmJi1eDecN0w+eebAL7G3Zylz0EFlPbdpL1KuUOgj5Nr/fvu+zGz7fX4\n\tRpV+yLINKP9GdB9gUAtfvs22I5Tsax9vGNyG/SOJnRxfCIJ30lE5tI3Bf+Ua/gBmeFzu\n\tOKzsJzyCLEpoP0hc2ELzlLxjZBaYm/qIl5IV5b4g1R9b4Ap4Zb4av9R33NAhUWm8blGQ\n\tn91w1BUZIloluG91QRsY9MNJnmS2rlNwGjrZuqh5hAJI1G92u19luXJRZtgTg268+lsF\n\tU9aw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:subject:message-id:mime-version\n\t:content-disposition:user-agent;\n\tbh=y4lW4sfL0CUAZ4TNRnAc1oO9jlALN1Z/xI/HJ0V+T2k=;\n\tb=ovbM8z/VAb/in6tmjpH7+xwnYYFfV2LKRg042vMrLrT1Gre0KDmkkB/jtkjrAZxJE3\n\txEy4qSFdBhYc68hDyd6l9O8vbJ23Nm71p0KxPgw3x4PwwAYVSXQu8K3vlgOWKZ+ygG2X\n\tWJlAuaQNr8rKZ0jHlCnE9RsKgNEQtNDd80IAETlkbNtzwgCBH1b4g9ppWdRr5XziwrWR\n\tIKkNcZhIPiCSRDbu9d/aTpEEom9tHGpiMl8oX6R1XQWHar5D3TxronmhCGcSY6Zx4cFa\n\tnoaJN/Cy7Yvgs5hyQ3ZIAwIdVLmFcP2YXSafRgoNc+bhZaZgYTlDlGhAFXr8MnF8tmuL\n\tBQTw==","X-Gm-Message-State":"AOAM5310YCQnCBthWvjRhDwQc6HdYvZaymzKqMxB9Zu4rrKpmKKzJtQt\n\tG+GbkV/+XCtdRf9u0I4EOkt8dHGDWbg=","X-Google-Smtp-Source":"ABdhPJyIv3WuXGsTcJSRJFTTjpgiv6zPoJgElpo4dM06/2nW3Hb68xVvGMefxNaf4URT6VC4M3KjwA==","X-Received":"by 2002:a17:90b:358e:: with SMTP id\n\tmm14mr10664764pjb.54.1595425389612; \n\tWed, 22 Jul 2020 06:43:09 -0700 (PDT)","Date":"Wed, 22 Jul 2020 19:13:04 +0530","From":"Kaaira Gupta <kgupta@es.iitr.ac.in>","To":"libcamera-devel@lists.libcamera.org,\n\tKieran Bingham <kieran.bingham@ideasonboard.com>","Message-ID":"<20200722134303.GA27212@kaaira-HP-Pavilion-Notebook>","MIME-Version":"1.0","Content-Disposition":"inline","User-Agent":"Mutt/1.9.4 (2018-02-28)","Subject":"[libcamera-devel] media: vimc: VIMC multiple streams","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"},"content":"This is a patchset by Niklas (https://patchwork.kernel.org/cover/10948831/)\nthat I have rebased on the current media-tree, I have not added anything\nto it and it is just a compressed version of his patchset into a single\npatch, based on the latest master to aid testing multiple stream support\nin the VIMC pipeline.\n\nSigned-off-by: Kaaira Gupta <kgupta@es.iitr.ac.in>\n---\n .../media/test-drivers/vimc/vimc-capture.c    | 35 ++++++++++++++++++-\n .../media/test-drivers/vimc/vimc-debayer.c    |  8 +++++\n drivers/media/test-drivers/vimc/vimc-scaler.c |  8 +++++\n drivers/media/test-drivers/vimc/vimc-sensor.c |  8 +++++\n .../media/test-drivers/vimc/vimc-streamer.c   | 22 +++++++-----\n 5 files changed, 71 insertions(+), 10 deletions(-)","diff":"diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c\nindex c63496b17b9a..5fc56582af83 100644\n--- a/drivers/media/test-drivers/vimc/vimc-capture.c\n+++ b/drivers/media/test-drivers/vimc/vimc-capture.c\n@@ -237,16 +237,49 @@ static void vimc_cap_return_all_buffers(struct vimc_cap_device *vcap,\n \tspin_unlock(&vcap->qlock);\n }\n \n+static struct media_entity *vimc_cap_get_sensor(struct vimc_cap_device *vcap)\n+{\n+\tstruct media_entity *entity = &vcap->vdev.entity;\n+\tstruct media_device *mdev = entity->graph_obj.mdev;\n+\tstruct media_graph graph;\n+\n+\tmutex_lock(&mdev->graph_mutex);\n+\tif (media_graph_walk_init(&graph, mdev)) {\n+\t\tmutex_unlock(&mdev->graph_mutex);\n+\t\treturn NULL;\n+\t}\n+\n+\tmedia_graph_walk_start(&graph, entity);\n+\n+\twhile ((entity = media_graph_walk_next(&graph)))\n+\t\tif (entity->function == MEDIA_ENT_F_CAM_SENSOR)\n+\t\t\tbreak;\n+\n+\tmutex_unlock(&mdev->graph_mutex);\n+\n+\tmedia_graph_walk_cleanup(&graph);\n+\n+\treturn entity;\n+}\n+\n static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)\n {\n \tstruct vimc_cap_device *vcap = vb2_get_drv_priv(vq);\n \tstruct media_entity *entity = &vcap->vdev.entity;\n+\tstruct media_pipeline *pipe = NULL;\n+\tstruct media_entity *sensorent;\n \tint ret;\n \n \tvcap->sequence = 0;\n \n \t/* Start the media pipeline */\n-\tret = media_pipeline_start(entity, &vcap->stream.pipe);\n+\tsensorent = vimc_cap_get_sensor(vcap);\n+\tif (sensorent && sensorent->pipe)\n+\t\tpipe = sensorent->pipe;\n+\telse\n+\t\tpipe = &vcap->stream.pipe;\n+\n+\tret = media_pipeline_start(entity, pipe);\n \tif (ret) {\n \t\tvimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);\n \t\treturn ret;\ndiff --git a/drivers/media/test-drivers/vimc/vimc-debayer.c b/drivers/media/test-drivers/vimc/vimc-debayer.c\nindex c3f6fef34f68..93fe19d8d2b4 100644\n--- a/drivers/media/test-drivers/vimc/vimc-debayer.c\n+++ b/drivers/media/test-drivers/vimc/vimc-debayer.c\n@@ -29,6 +29,7 @@ struct vimc_deb_pix_map {\n struct vimc_deb_device {\n \tstruct vimc_ent_device ved;\n \tstruct v4l2_subdev sd;\n+\tatomic_t use_count;\n \t/* The active format */\n \tstruct v4l2_mbus_framefmt sink_fmt;\n \tu32 src_code;\n@@ -343,6 +344,9 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable)\n \t\tconst struct vimc_pix_map *vpix;\n \t\tunsigned int frame_size;\n \n+\t\tif (atomic_inc_return(&vdeb->use_count) != 1)\n+\t\t\treturn 0;\n+\n \t\tif (vdeb->src_frame)\n \t\t\treturn 0;\n \n@@ -368,6 +372,9 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable)\n \t\t\treturn -ENOMEM;\n \n \t} else {\n+\t\tif (atomic_dec_return(&vdeb->use_count) != 0)\n+\t\t\treturn 0;\n+\n \t\tif (!vdeb->src_frame)\n \t\t\treturn 0;\n \n@@ -595,6 +602,7 @@ static struct vimc_ent_device *vimc_deb_add(struct vimc_device *vimc,\n \tvdeb->ved.process_frame = vimc_deb_process_frame;\n \tvdeb->ved.dev = vimc->mdev.dev;\n \tvdeb->mean_win_size = vimc_deb_ctrl_mean_win_size.def;\n+\tatomic_set(&vdeb->use_count, 0);\n \n \t/* Initialize the frame format */\n \tvdeb->sink_fmt = sink_fmt_default;\ndiff --git a/drivers/media/test-drivers/vimc/vimc-scaler.c b/drivers/media/test-drivers/vimc/vimc-scaler.c\nindex 121fa7d62a2e..9b8458dbe57c 100644\n--- a/drivers/media/test-drivers/vimc/vimc-scaler.c\n+++ b/drivers/media/test-drivers/vimc/vimc-scaler.c\n@@ -25,6 +25,7 @@ MODULE_PARM_DESC(sca_mult, \" the image size multiplier\");\n struct vimc_sca_device {\n \tstruct vimc_ent_device ved;\n \tstruct v4l2_subdev sd;\n+\tatomic_t use_count;\n \t/* NOTE: the source fmt is the same as the sink\n \t * with the width and hight multiplied by mult\n \t */\n@@ -340,6 +341,9 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable)\n \t\tconst struct vimc_pix_map *vpix;\n \t\tunsigned int frame_size;\n \n+\t\tif (atomic_inc_return(&vsca->use_count) != 1)\n+\t\t\treturn 0;\n+\n \t\tif (vsca->src_frame)\n \t\t\treturn 0;\n \n@@ -363,6 +367,9 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable)\n \t\t\treturn -ENOMEM;\n \n \t} else {\n+\t\tif (atomic_dec_return(&vsca->use_count) != 0)\n+\t\t\treturn 0;\n+\n \t\tif (!vsca->src_frame)\n \t\t\treturn 0;\n \n@@ -506,6 +513,7 @@ static struct vimc_ent_device *vimc_sca_add(struct vimc_device *vimc,\n \n \tvsca->ved.process_frame = vimc_sca_process_frame;\n \tvsca->ved.dev = vimc->mdev.dev;\n+\tatomic_set(&vsca->use_count, 0);\n \n \t/* Initialize the frame format */\n \tvsca->sink_fmt = sink_fmt_default;\ndiff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c\nindex a2f09ac9a360..d68113886290 100644\n--- a/drivers/media/test-drivers/vimc/vimc-sensor.c\n+++ b/drivers/media/test-drivers/vimc/vimc-sensor.c\n@@ -18,6 +18,7 @@ struct vimc_sen_device {\n \tstruct vimc_ent_device ved;\n \tstruct v4l2_subdev sd;\n \tstruct tpg_data tpg;\n+\tatomic_t use_count;\n \tu8 *frame;\n \t/* The active format */\n \tstruct v4l2_mbus_framefmt mbus_format;\n@@ -201,6 +202,9 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)\n \t\tconst struct vimc_pix_map *vpix;\n \t\tunsigned int frame_size;\n \n+\t\tif (atomic_inc_return(&vsen->use_count) != 1)\n+\t\t\treturn 0;\n+\n \t\t/* Calculate the frame size */\n \t\tvpix = vimc_pix_map_by_code(vsen->mbus_format.code);\n \t\tframe_size = vsen->mbus_format.width * vpix->bpp *\n@@ -219,6 +223,9 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)\n \n \t} else {\n \n+\t\tif (atomic_dec_return(&vsen->use_count) != 0)\n+\t\t\treturn 0;\n+\n \t\tvfree(vsen->frame);\n \t\tvsen->frame = NULL;\n \t}\n@@ -359,6 +366,7 @@ static struct vimc_ent_device *vimc_sen_add(struct vimc_device *vimc,\n \n \tvsen->ved.process_frame = vimc_sen_process_frame;\n \tvsen->ved.dev = vimc->mdev.dev;\n+\tatomic_set(&vsen->use_count, 0);\n \n \t/* Initialize the frame format */\n \tvsen->mbus_format = fmt_default;\ndiff --git a/drivers/media/test-drivers/vimc/vimc-streamer.c b/drivers/media/test-drivers/vimc/vimc-streamer.c\nindex 451a32c0d034..d5b966a99f7b 100644\n--- a/drivers/media/test-drivers/vimc/vimc-streamer.c\n+++ b/drivers/media/test-drivers/vimc/vimc-streamer.c\n@@ -192,33 +192,35 @@ int vimc_streamer_s_stream(struct vimc_stream *stream,\n \t\t\t   struct vimc_ent_device *ved,\n \t\t\t   int enable)\n {\n+\tstatic DEFINE_MUTEX(vimc_streamer_lock);\n \tint ret;\n \n \tif (!stream || !ved)\n \t\treturn -EINVAL;\n \n+\tret = mutex_lock_interruptible(&vimc_streamer_lock);\n+\tif (ret)\n+\t\treturn ret;\n+\n \tif (enable) {\n \t\tif (stream->kthread)\n-\t\t\treturn 0;\n+\t\t\tgoto out;\n \n \t\tret = vimc_streamer_pipeline_init(stream, ved);\n \t\tif (ret)\n-\t\t\treturn ret;\n+\t\t\tgoto out;\n \n \t\tstream->kthread = kthread_run(vimc_streamer_thread, stream,\n \t\t\t\t\t      \"vimc-streamer thread\");\n \n \t\tif (IS_ERR(stream->kthread)) {\n \t\t\tret = PTR_ERR(stream->kthread);\n-\t\t\tdev_err(ved->dev, \"kthread_run failed with %d\\n\", ret);\n-\t\t\tvimc_streamer_pipeline_terminate(stream);\n-\t\t\tstream->kthread = NULL;\n-\t\t\treturn ret;\n+\t\t\tgoto out;\n \t\t}\n \n \t} else {\n \t\tif (!stream->kthread)\n-\t\t\treturn 0;\n+\t\t\tgoto out;\n \n \t\tret = kthread_stop(stream->kthread);\n \t\t/*\n@@ -228,12 +230,14 @@ int vimc_streamer_s_stream(struct vimc_stream *stream,\n \t\t * pipeline.\n \t\t */\n \t\tif (ret)\n-\t\t\tdev_dbg(ved->dev, \"kthread_stop returned '%d'\\n\", ret);\n+\t\t\tgoto out;\n \n \t\tstream->kthread = NULL;\n \n \t\tvimc_streamer_pipeline_terminate(stream);\n \t}\n+out:\n+\tmutex_unlock(&vimc_streamer_lock);\n \n-\treturn 0;\n+\treturn ret;\n }\n","prefixes":["libcamera-devel"]}