[{"id":20149,"web_url":"https://patchwork.libcamera.org/comment/20149/","msgid":"","date":"2021-10-12T20:05:00","subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nThank you for the patches.\n\nOn Tue, Oct 12, 2021 at 04:24:08PM +0100, Naushir Patuck wrote:\n> Hi,\n> \n> This set of changes have come about after a discusion on an issue raised on the Pi\n> libcamera-apps github repo [1]. From my little understanding, various distributions\n> use different methods to build and package libraries. As part of this process,\n> they may build these libraries outside of the upstream tree, and perhaps not\n> even in a git repo tree. If this happens, the version string generated during the\n> libcamera build is either empty or (if it is from a downstream tree) useless.\n> \n> This change allows the user to override the SHA value with a string passed into\n> the meson build options that would be used in-place of the one generated by the\n> gen-version.sh script. This would allow out-of-tree builds to provide a sensible\n> sha version string based off the upstream tree.\n> \n> I'm not too sure if this is the best way to do this, but it is a simple solution.\n> If anyone has other suggestions how we can overcome this, please do let me know.\n\nWe're stepping in the wonderful world of release management :-)\n\nWe'll certainly need to support builds from release tarballs, that's how\nmost distributions will handle it. We've mostly ignored the issue until\nnow as we have no tagged release, so there was no urgency, but it\ndoesn't mean we have to wait before solving this problem.\n\nOnce we'll have releases, we will have a version number stored in the\ntop-level meson.build file. The gen-version.sh script will return an\nempty string, so only the release version will be used. That's been\nassumed so far to cover all the needs, as if there's an official\nrelease, tags and branches in the repository can be used to find the\nexact commit that was used to generate the tarball.\n\nWhat we haven't considered is builds from tarballs that do not\ncorrespond to official releases. In those cases, the binary will report\nthe version number of the last tag, without any sign of local changes. I\nwonder if we could improve this by automating SHA handling in that case,\nby automatically adding a file to the tarball when running 'meson dist'\nwith the git commit ID. That file would be read by gen-version.sh if\npresent. That way, generated tarballs will point to a particular commit,\nand the process should be less error-prone as there won't be any need to\nset a meson option manually.\n\nThere are probably a few things details to be figured out (for instance,\nif the HEAD commit corresponds to a release tag, we likely want to skip\ninclusion of the SHA1, so maybe the commit ID file should actually be a\n.version file that stores the output of gen-version.sh), but what do you\nthink of the idea overall ?\n\n> [1]: https://github.com/raspberrypi/libcamera-apps/issues/122\n> \n> Naushir Patuck (2):\n> utils: Add an option to override SHA string in gen-version.sh\n> build: Add a \"version_sha\" meson build option\n> \n> meson.build | 4 +++-\n> meson_options.txt | 5 +++++\n> src/libcamera/meson.build | 3 ++-\n> utils/gen-version.sh | 25 ++++++++++++++++++-------\n> 4 files changed, 28 insertions(+), 9 deletions(-)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 15D87C323E\n\tfor ;\n\tTue, 12 Oct 2021 20:05:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8066F68F4D;\n\tTue, 12 Oct 2021 22:05:17 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D9DB368F4C\n\tfor ;\n\tTue, 12 Oct 2021 22:05:15 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 52EEAF1;\n\tTue, 12 Oct 2021 22:05:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"hskBUFbx\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1634069115;\n\tbh=yMOzrCqBQTQTcoyA5+YjTzvlQL9AXW5Q6BLSJtRShQ0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=hskBUFbxHUP+Bdik1B9h8EZMPbwn/KcqhyxFFQ6Om3WtBG9JnZaOFhFMnvVyh/6OQ\n\t8qM0v6viqmwOUAZsQU9vAkueTCW0pmc+OlQg8vp1ojvzwJm3Rb/kSLLhz3p8/RwH2Z\n\tVR2/xaV0TeV68a7kwi6p+YgLuQBaKOcazty67TSE=","Date":"Tue, 12 Oct 2021 23:05:00 +0300","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20211012152410.978077-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20211012152410.978077-1-naush@raspberrypi.com>","Subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20150,"web_url":"https://patchwork.libcamera.org/comment/20150/","msgid":"","date":"2021-10-12T20:30:17","subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Oct 12, 2021 at 11:05:00PM +0300, Laurent Pinchart wrote:\n> Hi Naush,\n> \n> Thank you for the patches.\n> \n> On Tue, Oct 12, 2021 at 04:24:08PM +0100, Naushir Patuck wrote:\n> > Hi,\n> > \n> > This set of changes have come about after a discusion on an issue raised on the Pi\n> > libcamera-apps github repo [1]. From my little understanding, various distributions\n> > use different methods to build and package libraries. As part of this process,\n> > they may build these libraries outside of the upstream tree, and perhaps not\n> > even in a git repo tree. If this happens, the version string generated during the\n> > libcamera build is either empty or (if it is from a downstream tree) useless.\n> > \n> > This change allows the user to override the SHA value with a string passed into\n> > the meson build options that would be used in-place of the one generated by the\n> > gen-version.sh script. This would allow out-of-tree builds to provide a sensible\n> > sha version string based off the upstream tree.\n> > \n> > I'm not too sure if this is the best way to do this, but it is a simple solution.\n> > If anyone has other suggestions how we can overcome this, please do let me know.\n> \n> We're stepping in the wonderful world of release management :-)\n> \n> We'll certainly need to support builds from release tarballs, that's how\n> most distributions will handle it. We've mostly ignored the issue until\n> now as we have no tagged release, so there was no urgency, but it\n> doesn't mean we have to wait before solving this problem.\n> \n> Once we'll have releases, we will have a version number stored in the\n> top-level meson.build file. The gen-version.sh script will return an\n> empty string, so only the release version will be used. That's been\n> assumed so far to cover all the needs, as if there's an official\n> release, tags and branches in the repository can be used to find the\n> exact commit that was used to generate the tarball.\n> \n> What we haven't considered is builds from tarballs that do not\n> correspond to official releases. In those cases, the binary will report\n> the version number of the last tag, without any sign of local changes. I\n> wonder if we could improve this by automating SHA handling in that case,\n> by automatically adding a file to the tarball when running 'meson dist'\n> with the git commit ID. That file would be read by gen-version.sh if\n> present. That way, generated tarballs will point to a particular commit,\n> and the process should be less error-prone as there won't be any need to\n> set a meson option manually.\n> \n> There are probably a few things details to be figured out (for instance,\n> if the HEAD commit corresponds to a release tag, we likely want to skip\n> inclusion of the SHA1, so maybe the commit ID file should actually be a\n> .version file that stores the output of gen-version.sh), but what do you\n> think of the idea overall ?\n\nThis could be implemented with a new script in utils/ that gets run by\n`meson dist` through meson.add_dist_script() ([1]).\n\n[1] https://mesonbuild.com/Reference-manual_builtin_meson.html#mesonadd_dist_script\n\n> > [1]: https://github.com/raspberrypi/libcamera-apps/issues/122\n> > \n> > Naushir Patuck (2):\n> > utils: Add an option to override SHA string in gen-version.sh\n> > build: Add a \"version_sha\" meson build option\n> > \n> > meson.build | 4 +++-\n> > meson_options.txt | 5 +++++\n> > src/libcamera/meson.build | 3 ++-\n> > utils/gen-version.sh | 25 ++++++++++++++++++-------\n> > 4 files changed, 28 insertions(+), 9 deletions(-)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 40CE6BDC71\n\tfor ;\n\tTue, 12 Oct 2021 20:30:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AE03D68F4F;\n\tTue, 12 Oct 2021 22:30:33 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8717368F4C\n\tfor ;\n\tTue, 12 Oct 2021 22:30:31 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0D09C8C4;\n\tTue, 12 Oct 2021 22:30:30 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fLI8ko5P\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1634070631;\n\tbh=ukYPKNJPtuRrao3HfadWW05pPmlw08vwqKaOfXzIze8=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=fLI8ko5PwT1NX/juNDkFVbVO5ksbx9hG5AOLTdQ+u4nWnLzFFRomc1Bb7e99KnG1j\n\t8Ki0014twyB/vguUuOZhytHgAwxs+lSup7rvHmBogwAtBllIAwqmduWgviYfTo4tIc\n\t0VvP59CrXriJLsH173hNxIK/nmnfF7ebE0TO+33k=","Date":"Tue, 12 Oct 2021 23:30:17 +0300","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20211012152410.978077-1-naush@raspberrypi.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20170,"web_url":"https://patchwork.libcamera.org/comment/20170/","msgid":"","date":"2021-10-13T09:48:17","subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Laurent,\n\nOn Tue, 12 Oct 2021 at 21:05, Laurent Pinchart <\nlaurent.pinchart@ideasonboard.com> wrote:\n\n> Hi Naush,\n>\n> Thank you for the patches.\n>\n> On Tue, Oct 12, 2021 at 04:24:08PM +0100, Naushir Patuck wrote:\n> > Hi,\n> >\n> > This set of changes have come about after a discusion on an issue raised\n> on the Pi\n> > libcamera-apps github repo [1]. From my little understanding, various\n> distributions\n> > use different methods to build and package libraries. As part of this\n> process,\n> > they may build these libraries outside of the upstream tree, and perhaps\n> not\n> > even in a git repo tree. If this happens, the version string generated\n> during the\n> > libcamera build is either empty or (if it is from a downstream tree)\n> useless.\n> >\n> > This change allows the user to override the SHA value with a string\n> passed into\n> > the meson build options that would be used in-place of the one generated\n> by the\n> > gen-version.sh script. This would allow out-of-tree builds to provide a\n> sensible\n> > sha version string based off the upstream tree.\n> >\n> > I'm not too sure if this is the best way to do this, but it is a simple\n> solution.\n> > If anyone has other suggestions how we can overcome this, please do let\n> me know.\n>\n> We're stepping in the wonderful world of release management :-)\n>\n> We'll certainly need to support builds from release tarballs, that's how\n> most distributions will handle it. We've mostly ignored the issue until\n> now as we have no tagged release, so there was no urgency, but it\n> doesn't mean we have to wait before solving this problem.\n>\n> Once we'll have releases, we will have a version number stored in the\n> top-level meson.build file. The gen-version.sh script will return an\n> empty string, so only the release version will be used. That's been\n> assumed so far to cover all the needs, as if there's an official\n> release, tags and branches in the repository can be used to find the\n> exact commit that was used to generate the tarball.\n>\n> What we haven't considered is builds from tarballs that do not\n> correspond to official releases. In those cases, the binary will report\n> the version number of the last tag, without any sign of local changes.\n\n\nThis is probably the most likely mode of release for RPi :)\n\n I wonder if we could improve this by automating SHA handling in that case,\n\n> by automatically adding a file to the tarball when running 'meson dist'\n> with the git commit ID. That file would be read by gen-version.sh if\n> present. That way, generated tarballs will point to a particular commit,\n> and the process should be less error-prone as there won't be any need to\n> set a meson option manually.\n>\n\nThanks for the tips! I have read up on meson dist and think I have\nsomething\nthat will suit our needs better. In summary, the meson dist will run a\nscript\nthat generates the version string (with utils/gen-version.sh) and saves it\ninto\nthe distro tarball. I then will need to update the top level meson.build to\nread the version from the version.gen file if it exists, otherwise generate\nit as normal.\n\nThe Filesystem modele in meson has a handy fs.read() function, but requires\nme to bump up the minimum meson version from 0.55 to 0.57 so I will\njust do a cat and pipe command to read the version string from the file.\n\nI will prepare a change and we can discuss it in more detail.\n\nRegards,\nNaush\n\n\n>\n> There are probably a few things details to be figured out (for instance,\n> if the HEAD commit corresponds to a release tag, we likely want to skip\n> inclusion of the SHA1, so maybe the commit ID file should actually be a\n> .version file that stores the output of gen-version.sh), but what do you\n> think of the idea overall ?\n>\n> > [1]: https://github.com/raspberrypi/libcamera-apps/issues/122\n> >\n> > Naushir Patuck (2):\n> > utils: Add an option to override SHA string in gen-version.sh\n> > build: Add a \"version_sha\" meson build option\n> >\n> > meson.build | 4 +++-\n> > meson_options.txt | 5 +++++\n> > src/libcamera/meson.build | 3 ++-\n> > utils/gen-version.sh | 25 ++++++++++++++++++-------\n> > 4 files changed, 28 insertions(+), 9 deletions(-)\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DABA0C323E\n\tfor ;\n\tWed, 13 Oct 2021 09:48:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5585468F4F;\n\tWed, 13 Oct 2021 11:48:36 +0200 (CEST)","from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com\n\t[IPv6:2a00:1450:4864:20::12c])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D100D68546\n\tfor ;\n\tWed, 13 Oct 2021 11:48:34 +0200 (CEST)","by mail-lf1-x12c.google.com with SMTP id u21so6004285lff.8\n\tfor ;\n\tWed, 13 Oct 2021 02:48:34 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"R7ckaq6J\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=qrZilP+OrjOVBLseTdbktEUjh77TOtEQDHG/Eyj3H24=;\n\tb=R7ckaq6J0M6TzdDjW52RS57FFN0/787AJpVHO+SZEqvZXuAfqRhsyc00Et49bEYuVu\n\tSiTwyi90AKXiTNKPF9roXA8wtQjHhaDwumO0nLyMchXBB3JxDCDqxnY3wHLIrp96fVMW\n\ty3JLAvczrVK+1LAYlvwkdf5Jg97HzQ6HW17+5dXIh9TGuh0xNTe08xe2/Ub2gNSKm/jG\n\t/LHEWEc5XYns4LW5VZPy24TyhtygECO9fgCyGIx+esR7tHQBT5lac4djNp5RB05QOSJH\n\tLYWYBaLnYt2oWzIpfx5GcjyQIl3BOROJalNKSEfMmBQNRhBBF26YgnMhfLHFz8oq9tha\n\t9JDg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=qrZilP+OrjOVBLseTdbktEUjh77TOtEQDHG/Eyj3H24=;\n\tb=7aY/56XOBsi1tjjvxwRsoVtqINDzcjylLgsWWfs7YXGvJRkQEGn+xpVigtITorBboI\n\tS5cnhBCRzr5YF6fIv+k9lG3fngO+NgqJfcaUFRehBgRNQuaPvK1as8gTvakT53jQi+8v\n\tlV8+4xajfXNX5CQbjSmsnq0pZoW4X3AMVAGblwhaeaJKhDhrmaFyHRJYNTA0hNdchY4L\n\tw6NUYNjSg42KMwioVEc0btg8zqGp1jaQA71JfdbibA3PUdAd4eNN4bWeIRKqUcx/izhe\n\tYbj6tAJUQUfpGszNcMuQpxjhcrfgh6syThSyQ5Q6rmqkIPVY+RXXQOwf9SfYmHPDIWW4\n\teL4A==","X-Gm-Message-State":"AOAM5320wwYhpy01dwbUFj+uLpSXgLEP38sClTs9Al80SZxu2rVFNrRq\n\tsSwR7BRFeqRYfU1USPwxP2xv7MIYz9h9eEnPIN9tFA==","X-Google-Smtp-Source":"ABdhPJwpKMn66d4hssjiMsQ0tJ3NtL7nKJdyKjG95CC3Qy45Y7szc9fMy44zpM7//qSlVulRqyIaR263cZejZXabATk=","X-Received":"by 2002:a05:6512:3341:: with SMTP id\n\ty1mr38541197lfd.687.1634118513909; \n\tWed, 13 Oct 2021 02:48:33 -0700 (PDT)","MIME-Version":"1.0","References":"<20211012152410.978077-1-naush@raspberrypi.com>\n\t","In-Reply-To":"","From":"Naushir Patuck ","Date":"Wed, 13 Oct 2021 10:48:17 +0100","Message-ID":"","To":"Laurent Pinchart ","Content-Type":"multipart/alternative; boundary=\"0000000000008af1a605ce38dd0c\"","Subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20173,"web_url":"https://patchwork.libcamera.org/comment/20173/","msgid":"","date":"2021-10-13T10:08:02","subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nOn Wed, Oct 13, 2021 at 10:48:17AM +0100, Naushir Patuck wrote:\n> On Tue, 12 Oct 2021 at 21:05, Laurent Pinchart wrote:\n> > On Tue, Oct 12, 2021 at 04:24:08PM +0100, Naushir Patuck wrote:\n> > > Hi,\n> > >\n> > > This set of changes have come about after a discusion on an issue raised on the Pi\n> > > libcamera-apps github repo [1]. From my little understanding, various distributions\n> > > use different methods to build and package libraries. As part of this process,\n> > > they may build these libraries outside of the upstream tree, and perhaps not\n> > > even in a git repo tree. If this happens, the version string generated during the\n> > > libcamera build is either empty or (if it is from a downstream tree) useless.\n> > >\n> > > This change allows the user to override the SHA value with a string passed into\n> > > the meson build options that would be used in-place of the one generated by the\n> > > gen-version.sh script. This would allow out-of-tree builds to provide a sensible\n> > > sha version string based off the upstream tree.\n> > >\n> > > I'm not too sure if this is the best way to do this, but it is a simple solution.\n> > > If anyone has other suggestions how we can overcome this, please do let me know.\n> >\n> > We're stepping in the wonderful world of release management :-)\n> >\n> > We'll certainly need to support builds from release tarballs, that's how\n> > most distributions will handle it. We've mostly ignored the issue until\n> > now as we have no tagged release, so there was no urgency, but it\n> > doesn't mean we have to wait before solving this problem.\n> >\n> > Once we'll have releases, we will have a version number stored in the\n> > top-level meson.build file. The gen-version.sh script will return an\n> > empty string, so only the release version will be used. That's been\n> > assumed so far to cover all the needs, as if there's an official\n> > release, tags and branches in the repository can be used to find the\n> > exact commit that was used to generate the tarball.\n> >\n> > What we haven't considered is builds from tarballs that do not\n> > correspond to official releases. In those cases, the binary will report\n> > the version number of the last tag, without any sign of local changes.\n> \n> This is probably the most likely mode of release for RPi :)\n> \n> I wonder if we could improve this by automating SHA handling in that case,\n> \n> > by automatically adding a file to the tarball when running 'meson dist'\n> > with the git commit ID. That file would be read by gen-version.sh if\n> > present. That way, generated tarballs will point to a particular commit,\n> > and the process should be less error-prone as there won't be any need to\n> > set a meson option manually.\n> \n> Thanks for the tips! I have read up on meson dist and think I have something\n> that will suit our needs better. In summary, the meson dist will run a script\n> that generates the version string (with utils/gen-version.sh) and saves it into\n> the distro tarball. I then will need to update the top level meson.build to\n> read the version from the version.gen file if it exists, otherwise generate\n> it as normal.\n> \n> The Filesystem modele in meson has a handy fs.read() function, but requires\n> me to bump up the minimum meson version from 0.55 to 0.57 so I will\n> just do a cat and pipe command to read the version string from the file.\n\nWe try not to depend on meson versions that are not available in the\nlatest stable release of the major distributions. In this case, I would\nthus prefer handling this through a script. Would it make sense to\nmodify gen-version.sh to read the version file when the directory is not\nunder git's control ?\n\n> I will prepare a change and we can discuss it in more detail.\n> \n> > There are probably a few things details to be figured out (for instance,\n> > if the HEAD commit corresponds to a release tag, we likely want to skip\n> > inclusion of the SHA1, so maybe the commit ID file should actually be a\n> > .version file that stores the output of gen-version.sh), but what do you\n> > think of the idea overall ?\n> >\n> > > [1]: https://github.com/raspberrypi/libcamera-apps/issues/122\n> > >\n> > > Naushir Patuck (2):\n> > > utils: Add an option to override SHA string in gen-version.sh\n> > > build: Add a \"version_sha\" meson build option\n> > >\n> > > meson.build | 4 +++-\n> > > meson_options.txt | 5 +++++\n> > > src/libcamera/meson.build | 3 ++-\n> > > utils/gen-version.sh | 25 ++++++++++++++++++-------\n> > > 4 files changed, 28 insertions(+), 9 deletions(-)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 82F47BDC71\n\tfor ;\n\tWed, 13 Oct 2021 10:08:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CD3AB68F4F;\n\tWed, 13 Oct 2021 12:08:18 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B4D2460501\n\tfor ;\n\tWed, 13 Oct 2021 12:08:16 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2B3C618FF;\n\tWed, 13 Oct 2021 12:08:16 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Lq37Pl5V\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1634119696;\n\tbh=xKk8/wBDB/tw9uiGAFUCtstY5vl6P7sLm7L3OVPjIiA=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Lq37Pl5VdB75EqV2J9e73412kBaoSg9wyt5s+LN7mPxk7lJ0WKzNYSVt1ru7iVnNH\n\t2QYhRTfckN2W1IAtB5F+Gs3Fi6Sq3gGSejxkiXpydsEgwoOlDdtPDg+1oGwlxOz2LX\n\tXuA/eo10hiRsRnDpLGOjtXCbdo1Bb3DlDVl2AqTo=","Date":"Wed, 13 Oct 2021 13:08:02 +0300","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20211012152410.978077-1-naush@raspberrypi.com>\n\t\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20174,"web_url":"https://patchwork.libcamera.org/comment/20174/","msgid":"","date":"2021-10-13T10:12:59","subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Laurent,\n\nOn Wed, 13 Oct 2021 at 11:08, Laurent Pinchart <\nlaurent.pinchart@ideasonboard.com> wrote:\n\n> Hi Naush,\n>\n> On Wed, Oct 13, 2021 at 10:48:17AM +0100, Naushir Patuck wrote:\n> > On Tue, 12 Oct 2021 at 21:05, Laurent Pinchart wrote:\n> > > On Tue, Oct 12, 2021 at 04:24:08PM +0100, Naushir Patuck wrote:\n> > > > Hi,\n> > > >\n> > > > This set of changes have come about after a discusion on an issue\n> raised on the Pi\n> > > > libcamera-apps github repo [1]. From my little understanding,\n> various distributions\n> > > > use different methods to build and package libraries. As part of\n> this process,\n> > > > they may build these libraries outside of the upstream tree, and\n> perhaps not\n> > > > even in a git repo tree. If this happens, the version string\n> generated during the\n> > > > libcamera build is either empty or (if it is from a downstream tree)\n> useless.\n> > > >\n> > > > This change allows the user to override the SHA value with a string\n> passed into\n> > > > the meson build options that would be used in-place of the one\n> generated by the\n> > > > gen-version.sh script. This would allow out-of-tree builds to\n> provide a sensible\n> > > > sha version string based off the upstream tree.\n> > > >\n> > > > I'm not too sure if this is the best way to do this, but it is a\n> simple solution.\n> > > > If anyone has other suggestions how we can overcome this, please do\n> let me know.\n> > >\n> > > We're stepping in the wonderful world of release management :-)\n> > >\n> > > We'll certainly need to support builds from release tarballs, that's\n> how\n> > > most distributions will handle it. We've mostly ignored the issue until\n> > > now as we have no tagged release, so there was no urgency, but it\n> > > doesn't mean we have to wait before solving this problem.\n> > >\n> > > Once we'll have releases, we will have a version number stored in the\n> > > top-level meson.build file. The gen-version.sh script will return an\n> > > empty string, so only the release version will be used. That's been\n> > > assumed so far to cover all the needs, as if there's an official\n> > > release, tags and branches in the repository can be used to find the\n> > > exact commit that was used to generate the tarball.\n> > >\n> > > What we haven't considered is builds from tarballs that do not\n> > > correspond to official releases. In those cases, the binary will report\n> > > the version number of the last tag, without any sign of local changes.\n> >\n> > This is probably the most likely mode of release for RPi :)\n> >\n> > I wonder if we could improve this by automating SHA handling in that\n> case,\n> >\n> > > by automatically adding a file to the tarball when running 'meson dist'\n> > > with the git commit ID. That file would be read by gen-version.sh if\n> > > present. That way, generated tarballs will point to a particular\n> commit,\n> > > and the process should be less error-prone as there won't be any need\n> to\n> > > set a meson option manually.\n> >\n> > Thanks for the tips! I have read up on meson dist and think I have\n> something\n> > that will suit our needs better. In summary, the meson dist will run a\n> script\n> > that generates the version string (with utils/gen-version.sh) and saves\n> it into\n> > the distro tarball. I then will need to update the top level\n> meson.build to\n> > read the version from the version.gen file if it exists, otherwise\n> generate\n> > it as normal.\n> >\n> > The Filesystem modele in meson has a handy fs.read() function, but\n> requires\n> > me to bump up the minimum meson version from 0.55 to 0.57 so I will\n> > just do a cat and pipe command to read the version string from the file.\n>\n> We try not to depend on meson versions that are not available in the\n> latest stable release of the major distributions.\n\n\nAgree, I don't want to bump up the version just to use that function.\n\n\n> In this case, I would\n> thus prefer handling this through a script. Would it make sense to\n> modify gen-version.sh to read the version file when the directory is not\n> under git's control ?\n>\n\nThis is possible, or I can simply do a run_command('cat ...').stdout() to\nread the file. I am about to post a RFC patch with the changes in the next\nfew minutes. We can discuss the implementation further there.\n\nRegards,\nNaush\n\n\n>\n> > I will prepare a change and we can discuss it in more detail.\n> >\n> > > There are probably a few things details to be figured out (for\n> instance,\n> > > if the HEAD commit corresponds to a release tag, we likely want to skip\n> > > inclusion of the SHA1, so maybe the commit ID file should actually be a\n> > > .version file that stores the output of gen-version.sh), but what do\n> you\n> > > think of the idea overall ?\n> > >\n> > > > [1]: https://github.com/raspberrypi/libcamera-apps/issues/122\n> > > >\n> > > > Naushir Patuck (2):\n> > > > utils: Add an option to override SHA string in gen-version.sh\n> > > > build: Add a \"version_sha\" meson build option\n> > > >\n> > > > meson.build | 4 +++-\n> > > > meson_options.txt | 5 +++++\n> > > > src/libcamera/meson.build | 3 ++-\n> > > > utils/gen-version.sh | 25 ++++++++++++++++++-------\n> > > > 4 files changed, 28 insertions(+), 9 deletions(-)\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 970F2C323E\n\tfor ;\n\tWed, 13 Oct 2021 10:13:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EB42768F51;\n\tWed, 13 Oct 2021 12:13:17 +0200 (CEST)","from mail-lf1-x134.google.com (mail-lf1-x134.google.com\n\t[IPv6:2a00:1450:4864:20::134])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A7EFF60501\n\tfor ;\n\tWed, 13 Oct 2021 12:13:16 +0200 (CEST)","by mail-lf1-x134.google.com with SMTP id i24so9345838lfj.13\n\tfor ;\n\tWed, 13 Oct 2021 03:13:16 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"q2frnuQa\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=eKRSNZwloGrN5QvnwIdiNMpQzFmgIlwNp7TJBiMDsug=;\n\tb=q2frnuQa4ysiQFBFDzZh91ACKUxlzdlCz6FnzwquJMKYHNEec/3RWASLp0rjjEgc5r\n\tvVZz8mll7q+0r0XZyHaFQWw3ke4QDvfEdU5+g7oInR5U7zBigDtKcAc/DA2em8JMdBIt\n\tHhFREQ8Foh4yFn23Le/JD0fCs7psVylE885A/8+o5N/TQzN1Nbh4FcmqzqSpd5YMPnDT\n\t4Xo0yFY7rRIGOz4kfKFkrHqwewpQ3uJfdlu1rAIHidnNlmTp8YCzm6v7bl0UO8aV1o2w\n\tfmNgDILIL9fpaRzgk74isSa6uMtSIIAdqzA6k+BLF0UgPWTURWYmjVY+yUTsac2NtVFg\n\tjvxg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=eKRSNZwloGrN5QvnwIdiNMpQzFmgIlwNp7TJBiMDsug=;\n\tb=stmKSwRZUdCT91CjfyMpaE7Y8RyqyXRFMiKKpi9sK2BCcQCiamxKw2x2exCz8EdsZ0\n\tzKbaN02k4h1iXDMkSqxqFYC8tVLZ46IcPsUz4wUAW6kwkUxXOO8jilfo5nt0Lsfwcjet\n\tVJ9ab7sXyXVPN2dKaN8VkvtFm8OknBxzufOj7skK928MG95tfJOSkC7MbwaHCywEMnJA\n\tmyfz2oh/m/qaFbKjdboKWx6NIoQfToMkRGBziaPgktgPlYdwk8zD4FOCp3+BBmKkW3uq\n\tDv99QuPQAezpASu5+6QxmDYiTi4OrOShlDW2b1XrNCfm5onxo7q1AQ3r5DvBgXF4Jj4C\n\tgWlg==","X-Gm-Message-State":"AOAM531YNnnNRTz0ujwm0Qo9xdGtpAQaIkpRBMcJjzaVkOI8d8EtZYhv\n\t4BONmtVQ2rwEln2Fpcxi2t+c8q4DwHhe4xkBCRbulSTkKwE=","X-Google-Smtp-Source":"ABdhPJy4wTmQzBKfQ366Apji4j9YfcssrhOH1RnoCkTxS/nsNxCJAStusK59g0VqQEgvwI3+42PBmHimuQG445uUdlw=","X-Received":"by 2002:a05:651c:1505:: with SMTP id\n\te5mr14170411ljf.520.1634119995491; \n\tWed, 13 Oct 2021 03:13:15 -0700 (PDT)","MIME-Version":"1.0","References":"<20211012152410.978077-1-naush@raspberrypi.com>\n\t\n\t\n\t","In-Reply-To":"","From":"Naushir Patuck ","Date":"Wed, 13 Oct 2021 11:12:59 +0100","Message-ID":"","To":"Laurent Pinchart ","Content-Type":"multipart/alternative; boundary=\"000000000000da1f9e05ce3935a7\"","Subject":"Re: [libcamera-devel] [RFC 0/2] Add new build option to override\n\tgenerate SHA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]