| Message ID | 20250616142455.3424226-1-julien.vuillaumier@nxp.com |
|---|---|
| State | New |
| Headers | show |
| Series |
|
| Related | show |
Hi 2025. 06. 16. 16:24 keltezéssel, Julien Vuillaumier írta: > Libcamera, when built for the Yocto distribution, has a spurious > `-dirty` suffix in the git version associated to the library. > > Git version and dirty tree detection are generated at compilation time > by the script gen-version.h, that eventually appends the `-dirty` > suffix. Dirty detection is done using the `git diff-index` command > that requires a prior `git update-index --refresh` to avoid false > positive. However the index refresh is currently done only for > in-source build. > > Yocto builder (bitbake) uses out-of-source build. In that context, the > false positives come from file timestamp change and hard link creation > in the source tree, done by some of the builder tasks. > > This changes is to have `git update-index --refresh` command executed > also in case of out-of-source build, to fix the false dirty > detections. This is essentially a revert of https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 ("libcamera: skip auto version generation when building for Chromium OS") What remains to be seen is how the two can be reconciled. Regards, Barnabás Pőcze > > Issue can be reproduced with commands: > git clone https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 > cd libcamera > build_dir=$(realpath ../build) > src_dir=$(realpath .) > ./utils/gen-version.sh "$build_dir" "src_dir" > (0.5.1) > touch README.rst > ./utils/gen-version.sh "$build_dir" "src_dir" > (0.5.1+dirty (2025-06-06T15:38:15CEST)) > > Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> > --- > utils/gen-version.sh | 5 +---- > 1 file changed, 1 insertion(+), 4 deletions(-) > > diff --git a/utils/gen-version.sh b/utils/gen-version.sh > index 1b818e9e..3f917ee8 100755 > --- a/utils/gen-version.sh > +++ b/utils/gen-version.sh > @@ -38,10 +38,7 @@ fi > > # Append a '-dirty' suffix if the working tree is dirty. Prevent false > # positives due to changed timestamps by running git update-index. > -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") > -then > - git update-index --refresh > /dev/null 2>&1 > -fi > +git update-index --refresh > /dev/null 2>&1 > git diff-index --quiet HEAD || version="$version-dirty ($(date +%Y-%m-%dT%H:%M:%S%Z))" > > # If a project version is provided, use it to replace the version number.
Hi Barnabás, On 16/06/2025 16:30, Barnabás Pőcze wrote: > 2025. 06. 16. 16:24 keltezéssel, Julien Vuillaumier írta: >> Libcamera, when built for the Yocto distribution, has a spurious >> `-dirty` suffix in the git version associated to the library. >> >> Git version and dirty tree detection are generated at compilation time >> by the script gen-version.h, that eventually appends the `-dirty` >> suffix. Dirty detection is done using the `git diff-index` command >> that requires a prior `git update-index --refresh` to avoid false >> positive. However the index refresh is currently done only for >> in-source build. >> >> Yocto builder (bitbake) uses out-of-source build. In that context, the >> false positives come from file timestamp change and hard link creation >> in the source tree, done by some of the builder tasks. >> >> This changes is to have `git update-index --refresh` command executed >> also in case of out-of-source build, to fix the false dirty >> detections. > > This is essentially a revert of > https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 > ("libcamera: skip auto version generation when building for Chromium OS") > What remains to be seen is how the two can be reconciled. This is correct, that ends up reverting the commit https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 It is unfortunate to have an issue with other distributions to handle that special case. I don't know about Chromium OS build system and if that change is still applicable though. There may also be an option to detect a Chromium OS build by testing an environment variable or such. Thanks, Julien >> >> Issue can be reproduced with commands: >> git clone >> https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 >> cd libcamera >> build_dir=$(realpath ../build) >> src_dir=$(realpath .) >> ./utils/gen-version.sh "$build_dir" "src_dir" >> (0.5.1) >> touch README.rst >> ./utils/gen-version.sh "$build_dir" "src_dir" >> (0.5.1+dirty (2025-06-06T15:38:15CEST)) >> >> Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> >> --- >> utils/gen-version.sh | 5 +---- >> 1 file changed, 1 insertion(+), 4 deletions(-) >> >> diff --git a/utils/gen-version.sh b/utils/gen-version.sh >> index 1b818e9e..3f917ee8 100755 >> --- a/utils/gen-version.sh >> +++ b/utils/gen-version.sh >> @@ -38,10 +38,7 @@ fi >> >> # Append a '-dirty' suffix if the working tree is dirty. Prevent false >> # positives due to changed timestamps by running git update-index. >> -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") >> -then >> - git update-index --refresh > /dev/null 2>&1 >> -fi >> +git update-index --refresh > /dev/null 2>&1 >> git diff-index --quiet HEAD || version="$version-dirty ($(date >> +%Y-%m-%dT%H:%M:%S%Z))" >> >> # If a project version is provided, use it to replace the version >> number. >
Hi Julien, On Tue, Jun 17, 2025 at 11:54:21AM +0200, Julien Vuillaumier wrote: > On 16/06/2025 16:30, Barnabás Pőcze wrote: > > 2025. 06. 16. 16:24 keltezéssel, Julien Vuillaumier írta: > >> Libcamera, when built for the Yocto distribution, has a spurious > >> `-dirty` suffix in the git version associated to the library. > >> > >> Git version and dirty tree detection are generated at compilation time > >> by the script gen-version.h, that eventually appends the `-dirty` > >> suffix. Dirty detection is done using the `git diff-index` command > >> that requires a prior `git update-index --refresh` to avoid false > >> positive. However the index refresh is currently done only for > >> in-source build. > >> > >> Yocto builder (bitbake) uses out-of-source build. In that context, the > >> false positives come from file timestamp change and hard link creation > >> in the source tree, done by some of the builder tasks. What do you mean by out-of-source builds here ? meson never builds in the source tree, so I'm a bit curious. I'm also curious if anyone else has run into this issue before, as there are quite a few people compiling libcamera in Yocto, and as far as I know the problem has never been reported. Kieran, is this something you've noticed ? > >> This changes is to have `git update-index --refresh` command executed > >> also in case of out-of-source build, to fix the false dirty > >> detections. > > > > This is essentially a revert of > > https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 > > ("libcamera: skip auto version generation when building for Chromium OS") > > What remains to be seen is how the two can be reconciled. > > This is correct, that ends up reverting the commit > https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 > > It is unfortunate to have an issue with other distributions to handle > that special case. I don't know about Chromium OS build system and if > that change is still applicable though. There may also be an option to > detect a Chromium OS build by testing an environment variable or such. > > >> Issue can be reproduced with commands: > >> git clone > >> https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 > >> cd libcamera > >> build_dir=$(realpath ../build) > >> src_dir=$(realpath .) > >> ./utils/gen-version.sh "$build_dir" "src_dir" > >> (0.5.1) > >> touch README.rst > >> ./utils/gen-version.sh "$build_dir" "src_dir" > >> (0.5.1+dirty (2025-06-06T15:38:15CEST)) > >> > >> Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> > >> --- > >> utils/gen-version.sh | 5 +---- > >> 1 file changed, 1 insertion(+), 4 deletions(-) > >> > >> diff --git a/utils/gen-version.sh b/utils/gen-version.sh > >> index 1b818e9e..3f917ee8 100755 > >> --- a/utils/gen-version.sh > >> +++ b/utils/gen-version.sh > >> @@ -38,10 +38,7 @@ fi > >> > >> # Append a '-dirty' suffix if the working tree is dirty. Prevent false > >> # positives due to changed timestamps by running git update-index. > >> -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") > >> -then > >> - git update-index --refresh > /dev/null 2>&1 > >> -fi > >> +git update-index --refresh > /dev/null 2>&1 > >> git diff-index --quiet HEAD || version="$version-dirty ($(date > >> +%Y-%m-%dT%H:%M:%S%Z))" > >> > >> # If a project version is provided, use it to replace the version > >> number.
Hi Laurent, On 18/06/2025 02:09, Laurent Pinchart wrote: > Hi Julien, > > On Tue, Jun 17, 2025 at 11:54:21AM +0200, Julien Vuillaumier wrote: >> On 16/06/2025 16:30, Barnabás Pőcze wrote: >>> 2025. 06. 16. 16:24 keltezéssel, Julien Vuillaumier írta: >>>> Libcamera, when built for the Yocto distribution, has a spurious >>>> `-dirty` suffix in the git version associated to the library. >>>> >>>> Git version and dirty tree detection are generated at compilation time >>>> by the script gen-version.h, that eventually appends the `-dirty` >>>> suffix. Dirty detection is done using the `git diff-index` command >>>> that requires a prior `git update-index --refresh` to avoid false >>>> positive. However the index refresh is currently done only for >>>> in-source build. >>>> >>>> Yocto builder (bitbake) uses out-of-source build. In that context, the >>>> false positives come from file timestamp change and hard link creation >>>> in the source tree, done by some of the builder tasks. > > What do you mean by out-of-source builds here ? meson never builds in > the source tree, so I'm a bit curious. I'm also curious if anyone else > has run into this issue before, as there are quite a few people > compiling libcamera in Yocto, and as far as I know the problem has never > been reported. By out-of-source build, I meant the case where the meson build directory is not located in the source tree. But I understand that the wording is misleading in that context. Then `gen-version.sh` script executes the `git update-index --refresh` command only when the meson build directory is within the source tree. That is not the case with a Yocto build, so that command is not executed. As of Yocto 5.2 (Walnascar), 2 kind of operation executed by the builder leave the tree out-of-sync, though unchanged: - unmatched in-place sed (no replacement done) - hard links creation > > Kieran, is this something you've noticed ? > >>>> This changes is to have `git update-index --refresh` command executed >>>> also in case of out-of-source build, to fix the false dirty >>>> detections. >>> >>> This is essentially a revert of >>> https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 >>> ("libcamera: skip auto version generation when building for Chromium OS") >>> What remains to be seen is how the two can be reconciled. >> >> This is correct, that ends up reverting the commit >> https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 >> >> It is unfortunate to have an issue with other distributions to handle >> that special case. I don't know about Chromium OS build system and if >> that change is still applicable though. There may also be an option to >> detect a Chromium OS build by testing an environment variable or such. >> >>>> Issue can be reproduced with commands: >>>> git clone >>>> https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 >>>> cd libcamera >>>> build_dir=$(realpath ../build) >>>> src_dir=$(realpath .) >>>> ./utils/gen-version.sh "$build_dir" "src_dir" >>>> (0.5.1) >>>> touch README.rst >>>> ./utils/gen-version.sh "$build_dir" "src_dir" >>>> (0.5.1+dirty (2025-06-06T15:38:15CEST)) >>>> >>>> Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> >>>> --- >>>> utils/gen-version.sh | 5 +---- >>>> 1 file changed, 1 insertion(+), 4 deletions(-) >>>> >>>> diff --git a/utils/gen-version.sh b/utils/gen-version.sh >>>> index 1b818e9e..3f917ee8 100755 >>>> --- a/utils/gen-version.sh >>>> +++ b/utils/gen-version.sh >>>> @@ -38,10 +38,7 @@ fi >>>> >>>> # Append a '-dirty' suffix if the working tree is dirty. Prevent false >>>> # positives due to changed timestamps by running git update-index. >>>> -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") >>>> -then >>>> - git update-index --refresh > /dev/null 2>&1 >>>> -fi >>>> +git update-index --refresh > /dev/null 2>&1 >>>> git diff-index --quiet HEAD || version="$version-dirty ($(date >>>> +%Y-%m-%dT%H:%M:%S%Z))" >>>> >>>> # If a project version is provided, use it to replace the version >>>> number. > > -- > Regards, > > Laurent Pinchart Thanks, Julien
Quoting Julien Vuillaumier (2025-06-18 09:51:06) > Hi Laurent, > > On 18/06/2025 02:09, Laurent Pinchart wrote: > > Hi Julien, > > > > On Tue, Jun 17, 2025 at 11:54:21AM +0200, Julien Vuillaumier wrote: > >> On 16/06/2025 16:30, Barnabás Pőcze wrote: > >>> 2025. 06. 16. 16:24 keltezéssel, Julien Vuillaumier írta: > >>>> Libcamera, when built for the Yocto distribution, has a spurious > >>>> `-dirty` suffix in the git version associated to the library. > >>>> > >>>> Git version and dirty tree detection are generated at compilation time > >>>> by the script gen-version.h, that eventually appends the `-dirty` > >>>> suffix. Dirty detection is done using the `git diff-index` command > >>>> that requires a prior `git update-index --refresh` to avoid false > >>>> positive. However the index refresh is currently done only for > >>>> in-source build. > >>>> > >>>> Yocto builder (bitbake) uses out-of-source build. In that context, the > >>>> false positives come from file timestamp change and hard link creation > >>>> in the source tree, done by some of the builder tasks. > > > > What do you mean by out-of-source builds here ? meson never builds in > > the source tree, so I'm a bit curious. I'm also curious if anyone else > > has run into this issue before, as there are quite a few people > > compiling libcamera in Yocto, and as far as I know the problem has never > > been reported. > > By out-of-source build, I meant the case where the meson build directory > is not located in the source tree. But I understand that the wording is > misleading in that context. > > Then `gen-version.sh` script executes the `git update-index --refresh` > command only when the meson build directory is within the source tree. > That is not the case with a Yocto build, so that command is not executed. > > As of Yocto 5.2 (Walnascar), 2 kind of operation executed by the builder > leave the tree out-of-sync, though unchanged: > - unmatched in-place sed (no replacement done) > - hard links creation We've just updated to Walnascar on our current project - but I haven't "looked" to see if there's an impact here (and my tree is always dirty anyway while I'm developing :D) I'll see if I can reproduce anything. -- Kieran > > > > > Kieran, is this something you've noticed ? > > > >>>> This changes is to have `git update-index --refresh` command executed > >>>> also in case of out-of-source build, to fix the false dirty > >>>> detections. > >>> > >>> This is essentially a revert of > >>> https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 > >>> ("libcamera: skip auto version generation when building for Chromium OS") > >>> What remains to be seen is how the two can be reconciled. > >> > >> This is correct, that ends up reverting the commit > >> https://gitlab.freedesktop.org/camera/libcamera/-/commit/10ec09025d6f52f2c1d0ba6b7e6943a603a386d9 > >> > >> It is unfortunate to have an issue with other distributions to handle > >> that special case. I don't know about Chromium OS build system and if > >> that change is still applicable though. There may also be an option to > >> detect a Chromium OS build by testing an environment variable or such. > >> > >>>> Issue can be reproduced with commands: > >>>> git clone > >>>> https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 > >>>> cd libcamera > >>>> build_dir=$(realpath ../build) > >>>> src_dir=$(realpath .) > >>>> ./utils/gen-version.sh "$build_dir" "src_dir" > >>>> (0.5.1) > >>>> touch README.rst > >>>> ./utils/gen-version.sh "$build_dir" "src_dir" > >>>> (0.5.1+dirty (2025-06-06T15:38:15CEST)) > >>>> > >>>> Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> > >>>> --- > >>>> utils/gen-version.sh | 5 +---- > >>>> 1 file changed, 1 insertion(+), 4 deletions(-) > >>>> > >>>> diff --git a/utils/gen-version.sh b/utils/gen-version.sh > >>>> index 1b818e9e..3f917ee8 100755 > >>>> --- a/utils/gen-version.sh > >>>> +++ b/utils/gen-version.sh > >>>> @@ -38,10 +38,7 @@ fi > >>>> > >>>> # Append a '-dirty' suffix if the working tree is dirty. Prevent false > >>>> # positives due to changed timestamps by running git update-index. > >>>> -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") > >>>> -then > >>>> - git update-index --refresh > /dev/null 2>&1 > >>>> -fi > >>>> +git update-index --refresh > /dev/null 2>&1 > >>>> git diff-index --quiet HEAD || version="$version-dirty ($(date > >>>> +%Y-%m-%dT%H:%M:%S%Z))" > >>>> > >>>> # If a project version is provided, use it to replace the version > >>>> number. > > > > -- > > Regards, > > > > Laurent Pinchart > > Thanks, > Julien > >
diff --git a/utils/gen-version.sh b/utils/gen-version.sh index 1b818e9e..3f917ee8 100755 --- a/utils/gen-version.sh +++ b/utils/gen-version.sh @@ -38,10 +38,7 @@ fi # Append a '-dirty' suffix if the working tree is dirty. Prevent false # positives due to changed timestamps by running git update-index. -if [ -z "$build_dir" ] || (echo "$build_dir" | grep -q "$src_dir") -then - git update-index --refresh > /dev/null 2>&1 -fi +git update-index --refresh > /dev/null 2>&1 git diff-index --quiet HEAD || version="$version-dirty ($(date +%Y-%m-%dT%H:%M:%S%Z))" # If a project version is provided, use it to replace the version number.
Libcamera, when built for the Yocto distribution, has a spurious `-dirty` suffix in the git version associated to the library. Git version and dirty tree detection are generated at compilation time by the script gen-version.h, that eventually appends the `-dirty` suffix. Dirty detection is done using the `git diff-index` command that requires a prior `git update-index --refresh` to avoid false positive. However the index refresh is currently done only for in-source build. Yocto builder (bitbake) uses out-of-source build. In that context, the false positives come from file timestamp change and hard link creation in the source tree, done by some of the builder tasks. This changes is to have `git update-index --refresh` command executed also in case of out-of-source build, to fix the false dirty detections. Issue can be reproduced with commands: git clone https://git.libcamera.org/libcamera/libcamera.git -b v0.5.1 cd libcamera build_dir=$(realpath ../build) src_dir=$(realpath .) ./utils/gen-version.sh "$build_dir" "src_dir" (0.5.1) touch README.rst ./utils/gen-version.sh "$build_dir" "src_dir" (0.5.1+dirty (2025-06-06T15:38:15CEST)) Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com> --- utils/gen-version.sh | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)