| Message ID | 20200727092108.6209-1-andrey.konovalov@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
Hi Andrey, On 27/07/2020 10:21, Andrey Konovalov wrote: > libcamera checks if RPATH or RUNPATH dynamic tag is present in > libcamera.so. If it does, it assumes that libcamera binaries are > run directly from the build directory without installing them, and > tries to use resorces like IPA modules from the build directory. > Mainline meson strips RPATH/RUNPATH out at install time (for > meson versions up to 0.54; the things are somewhat changed in 0.55). > But openembedded-core patches meson to disable RPATH/RUNPATH removal. > That's why we need to remove this tag manually in do_install_append(). Uh oh, what's changed... (I'll have to go take a look). - https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful If we're reliant upon meson behaviour which is no longer consistent, then we are going to have to do something else in libcamera. /me sighs ... > IPA module is signed (with openssl dgst) after it is built. But > during packaging the OE build system 1) splits out debugging info, > and 2) strips the binaries. So the IPA module *.so file installed > isn't the one which the signature was calculated against. Then > the signature check fails, and libcamera tries to run the IPA > module isolated (in a sandbox), which doesn't work if the IPA > module wasn't designed to run isolated. The easiest way to fix that > is to disable splitting out debug information and stripping the binaries > during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > INHIBIT_PACKAGE_STRIP. This sounds like an effective solution for openembedded, but it needs to be fixed in libcamera all the same. I'll try to follow up with the meson guys to see what we can do,. > Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > --- > .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > index 00a5c480d..573366f08 100644 > --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > > S = "${WORKDIR}/git" > > -DEPENDS = "python3-pyyaml-native udev gnutls boost" > +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > > RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > > inherit meson pkgconfig python3native > > +do_install_append() { > + chrpath -d ${D}${libdir}/libcamera.so Aha, I didn't know about chrpath, that looks helpful. Perhaps part of the solution will be handling our own strip/install actions to do this explicitly in the build. It will be a pain to have to pull in another external dependency though... > +} > + > FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > FILES_${PN} += " ${libdir}/libcamera.so" > > +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > +INHIBIT_PACKAGE_STRIP = "1" > + >
Hi Kieran, On 27.07.2020 12:42, Kieran Bingham wrote: > Hi Andrey, > > On 27/07/2020 10:21, Andrey Konovalov wrote: >> libcamera checks if RPATH or RUNPATH dynamic tag is present in >> libcamera.so. If it does, it assumes that libcamera binaries are >> run directly from the build directory without installing them, and >> tries to use resorces like IPA modules from the build directory. >> Mainline meson strips RPATH/RUNPATH out at install time (for >> meson versions up to 0.54; the things are somewhat changed in 0.55). >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >> That's why we need to remove this tag manually in do_install_append(). > > Uh oh, what's changed... (I'll have to go take a look). > > - > https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > > If we're reliant upon meson behaviour which is no longer consistent, > then we are going to have to do something else in libcamera. I haven't tried meson 0.55 yet, but my impression was that 0.55 should work just as before for "usual" (as per libcamera's README) libcamera build. And starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal *might* be dropped - if all the packages would be able to set RUNPATH to what they need, and meson would detect that OK in all those cases. Thanks, Andrey > /me sighs ... > >> IPA module is signed (with openssl dgst) after it is built. But >> during packaging the OE build system 1) splits out debugging info, >> and 2) strips the binaries. So the IPA module *.so file installed >> isn't the one which the signature was calculated against. Then >> the signature check fails, and libcamera tries to run the IPA >> module isolated (in a sandbox), which doesn't work if the IPA >> module wasn't designed to run isolated. The easiest way to fix that >> is to disable splitting out debug information and stripping the binaries >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >> INHIBIT_PACKAGE_STRIP. > > > This sounds like an effective solution for openembedded, but it needs to > be fixed in libcamera all the same. > > > I'll try to follow up with the meson guys to see what we can do,. > > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >> --- >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >> 1 file changed, 8 insertions(+), 1 deletion(-) >> >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> index 00a5c480d..573366f08 100644 >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >> >> S = "${WORKDIR}/git" >> >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >> >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >> >> inherit meson pkgconfig python3native >> >> +do_install_append() { >> + chrpath -d ${D}${libdir}/libcamera.so > > Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > the solution will be handling our own strip/install actions to do this > explicitly in the build. > > It will be a pain to have to pull in another external dependency though... > >> +} >> + >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >> FILES_${PN} += " ${libdir}/libcamera.so" >> >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >> +INHIBIT_PACKAGE_STRIP = "1" >> + >> >
Hello, On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: > On 27.07.2020 12:42, Kieran Bingham wrote: > > On 27/07/2020 10:21, Andrey Konovalov wrote: > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >> libcamera.so. If it does, it assumes that libcamera binaries are > >> run directly from the build directory without installing them, and > >> tries to use resorces like IPA modules from the build directory. > >> Mainline meson strips RPATH/RUNPATH out at install time (for > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >> That's why we need to remove this tag manually in do_install_append(). > > > > Uh oh, what's changed... (I'll have to go take a look). > > > > - > > https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > > > > If we're reliant upon meson behaviour which is no longer consistent, > > then we are going to have to do something else in libcamera. > > I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > just as before for "usual" (as per libcamera's README) libcamera build. And > starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > *might* be dropped - if all the packages would be able to set RUNPATH to > what they need, and meson would detect that OK in all those cases. I think that if the problem is caused by a meson patch in openembedded, then it would make sense to fix it there. We can decide to address the issue in libcamera itself if it's found to affect other distributions too, or if meson's behaviour changes in an incompatible way. > > /me sighs ... > > > >> IPA module is signed (with openssl dgst) after it is built. But > >> during packaging the OE build system 1) splits out debugging info, > >> and 2) strips the binaries. So the IPA module *.so file installed > >> isn't the one which the signature was calculated against. Then > >> the signature check fails, and libcamera tries to run the IPA > >> module isolated (in a sandbox), which doesn't work if the IPA > >> module wasn't designed to run isolated. The easiest way to fix that > >> is to disable splitting out debug information and stripping the binaries > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >> INHIBIT_PACKAGE_STRIP. > > > > This sounds like an effective solution for openembedded, but it needs to > > be fixed in libcamera all the same. > > > > I'll try to follow up with the meson guys to see what we can do,. We re-sign the IPA modules at install time for this very specific reason. If openembedded modifies the binaries after installing them, should it re-run the signing script ? > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >> --- > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >> 1 file changed, 8 insertions(+), 1 deletion(-) > >> > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> index 00a5c480d..573366f08 100644 > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >> > >> S = "${WORKDIR}/git" > >> > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >> > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >> > >> inherit meson pkgconfig python3native > >> > >> +do_install_append() { > >> + chrpath -d ${D}${libdir}/libcamera.so > > > > Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > > the solution will be handling our own strip/install actions to do this > > explicitly in the build. > > > > It will be a pain to have to pull in another external dependency though... > > > >> +} > >> + > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >> FILES_${PN} += " ${libdir}/libcamera.so" > >> > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >> +INHIBIT_PACKAGE_STRIP = "1" > >> +
On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov <andrey.konovalov@linaro.org> wrote: > > libcamera checks if RPATH or RUNPATH dynamic tag is present in > libcamera.so. If it does, it assumes that libcamera binaries are > run directly from the build directory without installing them, and > tries to use resorces like IPA modules from the build directory. > Mainline meson strips RPATH/RUNPATH out at install time (for > meson versions up to 0.54; the things are somewhat changed in 0.55). > But openembedded-core patches meson to disable RPATH/RUNPATH removal. > That's why we need to remove this tag manually in do_install_append(). > > IPA module is signed (with openssl dgst) after it is built. But > during packaging the OE build system 1) splits out debugging info, > and 2) strips the binaries. So the IPA module *.so file installed > isn't the one which the signature was calculated against. Then > the signature check fails, and libcamera tries to run the IPA > module isolated (in a sandbox), which doesn't work if the IPA > module wasn't designed to run isolated. The easiest way to fix that > is to disable splitting out debug information and stripping the binaries > during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > INHIBIT_PACKAGE_STRIP. > > Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > --- > .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > index 00a5c480d..573366f08 100644 > --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > > S = "${WORKDIR}/git" > > -DEPENDS = "python3-pyyaml-native udev gnutls boost" > +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > > RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > > inherit meson pkgconfig python3native > > +do_install_append() { > + chrpath -d ${D}${libdir}/libcamera.so > +} > + > FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > FILES_${PN} += " ${libdir}/libcamera.so" > > +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > +INHIBIT_PACKAGE_STRIP = "1" I think this is sub-optimal, it means we can not have stripped binaries and it will increase the size unnecessarily > + > -- > 2.17.1 >
On Mon, Jul 27, 2020 at 2:58 AM Andrey Konovalov <andrey.konovalov@linaro.org> wrote: > > Hi Kieran, > > On 27.07.2020 12:42, Kieran Bingham wrote: > > Hi Andrey, > > > > On 27/07/2020 10:21, Andrey Konovalov wrote: > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >> libcamera.so. If it does, it assumes that libcamera binaries are > >> run directly from the build directory without installing them, and > >> tries to use resorces like IPA modules from the build directory. > >> Mainline meson strips RPATH/RUNPATH out at install time (for > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >> That's why we need to remove this tag manually in do_install_append(). > > > > Uh oh, what's changed... (I'll have to go take a look). > > > > - > > https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > > > > If we're reliant upon meson behaviour which is no longer consistent, > > then we are going to have to do something else in libcamera. > > I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > just as before for "usual" (as per libcamera's README) libcamera build. And > starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > *might* be dropped - if all the packages would be able to set RUNPATH to > what they need, and meson would detect that OK in all those cases. runpath is problem more than solution when cross-compiling for a system which has normal file hierarchy standard since it gets absolute values of build time paths embedded into binaries, which is unwanted, so its stripped most of the time on OE. > > Thanks, > Andrey > > > /me sighs ... > > > >> IPA module is signed (with openssl dgst) after it is built. But > >> during packaging the OE build system 1) splits out debugging info, > >> and 2) strips the binaries. So the IPA module *.so file installed > >> isn't the one which the signature was calculated against. Then > >> the signature check fails, and libcamera tries to run the IPA > >> module isolated (in a sandbox), which doesn't work if the IPA > >> module wasn't designed to run isolated. The easiest way to fix that > >> is to disable splitting out debug information and stripping the binaries > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >> INHIBIT_PACKAGE_STRIP. > > > > > > This sounds like an effective solution for openembedded, but it needs to > > be fixed in libcamera all the same. > > > > > > I'll try to follow up with the meson guys to see what we can do,. > > > > > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >> --- > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >> 1 file changed, 8 insertions(+), 1 deletion(-) > >> > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> index 00a5c480d..573366f08 100644 > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >> > >> S = "${WORKDIR}/git" > >> > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >> > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >> > >> inherit meson pkgconfig python3native > >> > >> +do_install_append() { > >> + chrpath -d ${D}${libdir}/libcamera.so > > > > Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > > the solution will be handling our own strip/install actions to do this > > explicitly in the build. > > > > It will be a pain to have to pull in another external dependency though... > > > >> +} > >> + > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >> FILES_${PN} += " ${libdir}/libcamera.so" > >> > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >> +INHIBIT_PACKAGE_STRIP = "1" > >> + > >> > >
Hi Khem, On 27.07.2020 18:28, Khem Raj wrote: > On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov > <andrey.konovalov@linaro.org> wrote: >> >> libcamera checks if RPATH or RUNPATH dynamic tag is present in >> libcamera.so. If it does, it assumes that libcamera binaries are >> run directly from the build directory without installing them, and >> tries to use resorces like IPA modules from the build directory. >> Mainline meson strips RPATH/RUNPATH out at install time (for >> meson versions up to 0.54; the things are somewhat changed in 0.55). >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >> That's why we need to remove this tag manually in do_install_append(). >> >> IPA module is signed (with openssl dgst) after it is built. But >> during packaging the OE build system 1) splits out debugging info, >> and 2) strips the binaries. So the IPA module *.so file installed >> isn't the one which the signature was calculated against. Then >> the signature check fails, and libcamera tries to run the IPA >> module isolated (in a sandbox), which doesn't work if the IPA >> module wasn't designed to run isolated. The easiest way to fix that >> is to disable splitting out debug information and stripping the binaries >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >> INHIBIT_PACKAGE_STRIP. >> >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >> --- >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >> 1 file changed, 8 insertions(+), 1 deletion(-) >> >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> index 00a5c480d..573366f08 100644 >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >> >> S = "${WORKDIR}/git" >> >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >> >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >> >> inherit meson pkgconfig python3native >> >> +do_install_append() { >> + chrpath -d ${D}${libdir}/libcamera.so >> +} >> + >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >> FILES_${PN} += " ${libdir}/libcamera.so" >> >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >> +INHIBIT_PACKAGE_STRIP = "1" > > I think this is sub-optimal, it means we can not have stripped > binaries and it will increase the size unnecessarily Indeed. But the alternative is to recalculate the signature on the stripped binary in do_install_append(), and the drawback of this is moving part of the module signature implementation into the recipe. Or the libcamera implementation is to be changed to handle stripped binaries. Thanks, Andrey >> + >> -- >> 2.17.1 >>
On Mon, Jul 27, 2020 at 4:12 AM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hello, > > On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: > > On 27.07.2020 12:42, Kieran Bingham wrote: > > > On 27/07/2020 10:21, Andrey Konovalov wrote: > > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > > >> libcamera.so. If it does, it assumes that libcamera binaries are > > >> run directly from the build directory without installing them, and > > >> tries to use resorces like IPA modules from the build directory. > > >> Mainline meson strips RPATH/RUNPATH out at install time (for > > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > > >> That's why we need to remove this tag manually in do_install_append(). > > > > > > Uh oh, what's changed... (I'll have to go take a look). > > > > > > - > > > https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > > > > > > If we're reliant upon meson behaviour which is no longer consistent, > > > then we are going to have to do something else in libcamera. > > > > I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > > just as before for "usual" (as per libcamera's README) libcamera build. And > > starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > > *might* be dropped - if all the packages would be able to set RUNPATH to > > what they need, and meson would detect that OK in all those cases. > > I think that if the problem is caused by a meson patch in openembedded, > then it would make sense to fix it there. We can decide to address the > issue in libcamera itself if it's found to affect other distributions > too, or if meson's behaviour changes in an incompatible way. As I said in prior email, It causes problems in cross compiling, so perhaps it will be better to have an option to not specify it or reset it during configure. > > > > /me sighs ... > > > > > >> IPA module is signed (with openssl dgst) after it is built. But > > >> during packaging the OE build system 1) splits out debugging info, > > >> and 2) strips the binaries. So the IPA module *.so file installed > > >> isn't the one which the signature was calculated against. Then > > >> the signature check fails, and libcamera tries to run the IPA > > >> module isolated (in a sandbox), which doesn't work if the IPA > > >> module wasn't designed to run isolated. The easiest way to fix that > > >> is to disable splitting out debug information and stripping the binaries > > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > > >> INHIBIT_PACKAGE_STRIP. > > > > > > This sounds like an effective solution for openembedded, but it needs to > > > be fixed in libcamera all the same. > > > > > > I'll try to follow up with the meson guys to see what we can do,. > > We re-sign the IPA modules at install time for this very specific > reason. If openembedded modifies the binaries after installing them, > should it re-run the signing script ? build systems take on creating debuggable packages and for that usually, it builds the package and then takes the control of stripping the binaries since it will save the symbols and debug info into a separate package unlike install -s or explicit strip commands the components build system might do, which would discard this content unconditionally. Perhaps it would be better for libcamera buildsystem to take this into consideration in order for distros to be able to package it easily. so we need a way to resign it or not sign it at all since strip step runs past install during build. > > > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > > >> --- > > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > > >> 1 file changed, 8 insertions(+), 1 deletion(-) > > >> > > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > >> index 00a5c480d..573366f08 100644 > > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > > >> > > >> S = "${WORKDIR}/git" > > >> > > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > > >> > > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > > >> > > >> inherit meson pkgconfig python3native > > >> > > >> +do_install_append() { > > >> + chrpath -d ${D}${libdir}/libcamera.so > > > > > > Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > > > the solution will be handling our own strip/install actions to do this > > > explicitly in the build. > > > > > > It will be a pain to have to pull in another external dependency though... > > > > > >> +} > > >> + > > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > > >> FILES_${PN} += " ${libdir}/libcamera.so" > > >> > > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > > >> +INHIBIT_PACKAGE_STRIP = "1" > > >> + > > -- > Regards, > > Laurent Pinchart > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#85980): https://lists.openembedded.org/g/openembedded-devel/message/85980 > Mute This Topic: https://lists.openembedded.org/mt/75819340/1997914 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=-
On Mon, Jul 27, 2020 at 8:36 AM Andrey Konovalov <andrey.konovalov@linaro.org> wrote: > > Hi Khem, > > On 27.07.2020 18:28, Khem Raj wrote: > > On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov > > <andrey.konovalov@linaro.org> wrote: > >> > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >> libcamera.so. If it does, it assumes that libcamera binaries are > >> run directly from the build directory without installing them, and > >> tries to use resorces like IPA modules from the build directory. > >> Mainline meson strips RPATH/RUNPATH out at install time (for > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >> That's why we need to remove this tag manually in do_install_append(). > >> > >> IPA module is signed (with openssl dgst) after it is built. But > >> during packaging the OE build system 1) splits out debugging info, > >> and 2) strips the binaries. So the IPA module *.so file installed > >> isn't the one which the signature was calculated against. Then > >> the signature check fails, and libcamera tries to run the IPA > >> module isolated (in a sandbox), which doesn't work if the IPA > >> module wasn't designed to run isolated. The easiest way to fix that > >> is to disable splitting out debug information and stripping the binaries > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >> INHIBIT_PACKAGE_STRIP. > >> > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >> --- > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >> 1 file changed, 8 insertions(+), 1 deletion(-) > >> > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> index 00a5c480d..573366f08 100644 > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >> > >> S = "${WORKDIR}/git" > >> > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >> > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >> > >> inherit meson pkgconfig python3native > >> > >> +do_install_append() { > >> + chrpath -d ${D}${libdir}/libcamera.so > >> +} > >> + > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >> FILES_${PN} += " ${libdir}/libcamera.so" > >> > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >> +INHIBIT_PACKAGE_STRIP = "1" > > > > I think this is sub-optimal, it means we can not have stripped > > binaries and it will increase the size unnecessarily > > Indeed. > > But the alternative is to recalculate the signature on the stripped binary in do_install_append(), > and the drawback of this is moving part of the module signature implementation into the recipe. > > Or the libcamera implementation is to be changed to handle stripped binaries. > either of these solutions will be better perhaps. > Thanks, > Andrey > > >> + > >> -- > >> 2.17.1 > >>
Hi Khem, On Mon, Jul 27, 2020 at 08:37:27AM -0700, Khem Raj wrote: > On Mon, Jul 27, 2020 at 4:12 AM Laurent Pinchart wrote: > > On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: > > > On 27.07.2020 12:42, Kieran Bingham wrote: > > > > On 27/07/2020 10:21, Andrey Konovalov wrote: > > > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > > > >> libcamera.so. If it does, it assumes that libcamera binaries are > > > >> run directly from the build directory without installing them, and > > > >> tries to use resorces like IPA modules from the build directory. > > > >> Mainline meson strips RPATH/RUNPATH out at install time (for > > > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > > > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > > > >> That's why we need to remove this tag manually in do_install_append(). > > > > > > > > Uh oh, what's changed... (I'll have to go take a look). > > > > > > > > - > > > > https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > > > > > > > > If we're reliant upon meson behaviour which is no longer consistent, > > > > then we are going to have to do something else in libcamera. > > > > > > I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > > > just as before for "usual" (as per libcamera's README) libcamera build. And > > > starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > > > *might* be dropped - if all the packages would be able to set RUNPATH to > > > what they need, and meson would detect that OK in all those cases. > > > > I think that if the problem is caused by a meson patch in openembedded, > > then it would make sense to fix it there. We can decide to address the > > issue in libcamera itself if it's found to affect other distributions > > too, or if meson's behaviour changes in an incompatible way. > > As I said in prior email, It causes problems in cross compiling, so > perhaps it will be > better to have an option to not specify it or reset it during configure. That's something we can consider, yes. > > > > /me sighs ... > > > > > > > >> IPA module is signed (with openssl dgst) after it is built. But > > > >> during packaging the OE build system 1) splits out debugging info, > > > >> and 2) strips the binaries. So the IPA module *.so file installed > > > >> isn't the one which the signature was calculated against. Then > > > >> the signature check fails, and libcamera tries to run the IPA > > > >> module isolated (in a sandbox), which doesn't work if the IPA > > > >> module wasn't designed to run isolated. The easiest way to fix that > > > >> is to disable splitting out debug information and stripping the binaries > > > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > > > >> INHIBIT_PACKAGE_STRIP. > > > > > > > > This sounds like an effective solution for openembedded, but it needs to > > > > be fixed in libcamera all the same. > > > > > > > > I'll try to follow up with the meson guys to see what we can do,. > > > > We re-sign the IPA modules at install time for this very specific > > reason. If openembedded modifies the binaries after installing them, > > should it re-run the signing script ? > > build systems take on creating debuggable packages and for that > usually, it builds the package and then > takes the control of stripping the binaries since it will save the > symbols and debug info into a separate package > unlike install -s or explicit strip commands the components build > system might do, which would discard this > content unconditionally. Perhaps it would be better for libcamera > buildsystem to take this into consideration > in order for distros to be able to package it easily. so we need a way > to resign it or not sign it at all since strip > step runs past install during build. The src/ipa/ipa-sign-install.sh script can be used by the build system to regenerate signatures. It takes the private key file as the first parameter, followed by all the modules to be signed. We can improve the script to facilitate its integration with distribution packaging. > > > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > > > >> --- > > > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > > > >> 1 file changed, 8 insertions(+), 1 deletion(-) > > > >> > > > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > > >> index 00a5c480d..573366f08 100644 > > > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > > > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > > > >> > > > >> S = "${WORKDIR}/git" > > > >> > > > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > > > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > > > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > > > >> > > > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > > > >> > > > >> inherit meson pkgconfig python3native > > > >> > > > >> +do_install_append() { > > > >> + chrpath -d ${D}${libdir}/libcamera.so > > > > > > > > Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > > > > the solution will be handling our own strip/install actions to do this > > > > explicitly in the build. > > > > > > > > It will be a pain to have to pull in another external dependency though... > > > > > > > >> +} > > > >> + > > > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > > > >> FILES_${PN} += " ${libdir}/libcamera.so" > > > >> > > > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > > > >> +INHIBIT_PACKAGE_STRIP = "1" > > > >> +
Hi Andrey, On Mon, Jul 27, 2020 at 06:36:28PM +0300, Andrey Konovalov wrote: > On 27.07.2020 18:28, Khem Raj wrote: > > On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov wrote: > >> > >> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >> libcamera.so. If it does, it assumes that libcamera binaries are > >> run directly from the build directory without installing them, and > >> tries to use resorces like IPA modules from the build directory. > >> Mainline meson strips RPATH/RUNPATH out at install time (for > >> meson versions up to 0.54; the things are somewhat changed in 0.55). > >> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >> That's why we need to remove this tag manually in do_install_append(). > >> > >> IPA module is signed (with openssl dgst) after it is built. But > >> during packaging the OE build system 1) splits out debugging info, > >> and 2) strips the binaries. So the IPA module *.so file installed > >> isn't the one which the signature was calculated against. Then > >> the signature check fails, and libcamera tries to run the IPA > >> module isolated (in a sandbox), which doesn't work if the IPA > >> module wasn't designed to run isolated. The easiest way to fix that > >> is to disable splitting out debug information and stripping the binaries > >> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >> INHIBIT_PACKAGE_STRIP. > >> > >> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >> --- > >> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >> 1 file changed, 8 insertions(+), 1 deletion(-) > >> > >> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> index 00a5c480d..573366f08 100644 > >> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >> > >> S = "${WORKDIR}/git" > >> > >> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >> > >> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >> > >> inherit meson pkgconfig python3native > >> > >> +do_install_append() { > >> + chrpath -d ${D}${libdir}/libcamera.so > >> +} > >> + > >> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >> FILES_${PN} += " ${libdir}/libcamera.so" > >> > >> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >> +INHIBIT_PACKAGE_STRIP = "1" > > > > I think this is sub-optimal, it means we can not have stripped > > binaries and it will increase the size unnecessarily > > Indeed. > > But the alternative is to recalculate the signature on the stripped binary in do_install_append(), > and the drawback of this is moving part of the module signature implementation into the recipe. > > Or the libcamera implementation is to be changed to handle stripped binaries. We could calculate the signature on selected sections only, but that would make the implementation much more complex. Could the src/ipa/ipa-sign-install.sh resign script could be used by the recipe ? We can also improve the script to facilitate its usage. > >> +
Hi Laurent, On 27.07.2020 14:11, Laurent Pinchart wrote: > Hello, > > On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: >> On 27.07.2020 12:42, Kieran Bingham wrote: >>> On 27/07/2020 10:21, Andrey Konovalov wrote: >>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in >>>> libcamera.so. If it does, it assumes that libcamera binaries are >>>> run directly from the build directory without installing them, and >>>> tries to use resorces like IPA modules from the build directory. >>>> Mainline meson strips RPATH/RUNPATH out at install time (for >>>> meson versions up to 0.54; the things are somewhat changed in 0.55). >>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >>>> That's why we need to remove this tag manually in do_install_append(). >>> >>> Uh oh, what's changed... (I'll have to go take a look). >>> >>> - >>> https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful >>> >>> If we're reliant upon meson behaviour which is no longer consistent, >>> then we are going to have to do something else in libcamera. >> >> I haven't tried meson 0.55 yet, but my impression was that 0.55 should work >> just as before for "usual" (as per libcamera's README) libcamera build. And >> starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal >> *might* be dropped - if all the packages would be able to set RUNPATH to >> what they need, and meson would detect that OK in all those cases. > > I think that if the problem is caused by a meson patch in openembedded, > then it would make sense to fix it there. We can decide to address the > issue in libcamera itself if it's found to affect other distributions > too, or if meson's behaviour changes in an incompatible way. It looks like it is not openembedded only issue: -------- Forwarded Message -------- Subject: [libcamera-devel] [PATCH v4 0/2] package/libcamera: bump version to 96fab38 Date: Tue, 16 Jun 2020 20:59:49 +0200 From: Peter Seiderer <ps.report@gmx.net> To: buildroot@busybox.net CC: libcamera-devel@lists.libcamera.org, Yann E . MORIN <yann.morin.1998@free.fr> <snip> With the following patch libcamera is forced to believe it is running in a installed environment: diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp index d55338f..4ff9dac 100644 --- a/src/libcamera/utils.cpp +++ b/src/libcamera/utils.cpp @@ -346,15 +346,18 @@ details::StringSplitter split(const std::string &str, const std::string &delim) */ bool isLibcameraInstalled() { +#if 0 /* * DT_RUNPATH (DT_RPATH when the linker uses old dtags) is removed on * install. */ for (const ElfW(Dyn) *dyn = _DYNAMIC; dyn->d_tag != DT_NULL; ++dyn) { - if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) + if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) { + printf("XXXXX - dyn->d_un.d_ptr: %s\n", (char*)dyn->d_un.d_ptr); return false; + } } - +#endif return true; } Maybe this is because of the buildroot local meson patch ([1]), leading to an empty (but not absent) RPATH? <snip> [0:02:18.125804232] [252] DEBUG IPAManager ipa_manager.cpp:316 IPA module /usr/lib/libcamera/ipa_rpi.so signature is not valid <snip> This can be avoided with the following patch/hack (disable signature check): diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 505cf61..3d64898 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -301,6 +301,9 @@ std::unique_ptr<IPAProxy> IPAManager::createIPA(PipelineHandler *pipe, bool IPAManager::isSignatureValid(IPAModule *ipa) const { +#if 1 + return true; +#else #if HAVE_IPA_PUBKEY File file{ ipa->path() }; if (!file.open(File::ReadOnly)) @@ -320,6 +323,7 @@ bool IPAManager::isSignatureValid(IPAModule *ipa) const #else return false; #endif +#endif } } /* namespace libcamera */ Maybe related to the buildroot finalize and/or sanitizing RPATH in target tree step (and/or strip after install with BR2_ENABLE_DEBUG=y/BR2_STRIP_strip=y enabled)? -------- End of Forwarded Message -------- Thanks, Andrey >>> /me sighs ... >>> >>>> IPA module is signed (with openssl dgst) after it is built. But >>>> during packaging the OE build system 1) splits out debugging info, >>>> and 2) strips the binaries. So the IPA module *.so file installed >>>> isn't the one which the signature was calculated against. Then >>>> the signature check fails, and libcamera tries to run the IPA >>>> module isolated (in a sandbox), which doesn't work if the IPA >>>> module wasn't designed to run isolated. The easiest way to fix that >>>> is to disable splitting out debug information and stripping the binaries >>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >>>> INHIBIT_PACKAGE_STRIP. >>> >>> This sounds like an effective solution for openembedded, but it needs to >>> be fixed in libcamera all the same. >>> >>> I'll try to follow up with the meson guys to see what we can do,. > > We re-sign the IPA modules at install time for this very specific > reason. If openembedded modifies the binaries after installing them, > should it re-run the signing script ? > >>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >>>> --- >>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >>>> 1 file changed, 8 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> index 00a5c480d..573366f08 100644 >>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >>>> >>>> S = "${WORKDIR}/git" >>>> >>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >>>> >>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >>>> >>>> inherit meson pkgconfig python3native >>>> >>>> +do_install_append() { >>>> + chrpath -d ${D}${libdir}/libcamera.so >>> >>> Aha, I didn't know about chrpath, that looks helpful. Perhaps part of >>> the solution will be handling our own strip/install actions to do this >>> explicitly in the build. >>> >>> It will be a pain to have to pull in another external dependency though... >>> >>>> +} >>>> + >>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >>>> FILES_${PN} += " ${libdir}/libcamera.so" >>>> >>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >>>> +INHIBIT_PACKAGE_STRIP = "1" >>>> + > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#85980): https://lists.openembedded.org/g/openembedded-devel/message/85980 > Mute This Topic: https://lists.openembedded.org/mt/75819340/4454599 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [andrey.konovalov@linaro.org] > -=-=-=-=-=-=-=-=-=-=-=- >
Hi Laurent, On 27.07.2020 18:45, Laurent Pinchart wrote: > Hi Andrey, > > On Mon, Jul 27, 2020 at 06:36:28PM +0300, Andrey Konovalov wrote: >> On 27.07.2020 18:28, Khem Raj wrote: >>> On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov wrote: >>>> >>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in >>>> libcamera.so. If it does, it assumes that libcamera binaries are >>>> run directly from the build directory without installing them, and >>>> tries to use resorces like IPA modules from the build directory. >>>> Mainline meson strips RPATH/RUNPATH out at install time (for >>>> meson versions up to 0.54; the things are somewhat changed in 0.55). >>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >>>> That's why we need to remove this tag manually in do_install_append(). >>>> >>>> IPA module is signed (with openssl dgst) after it is built. But >>>> during packaging the OE build system 1) splits out debugging info, >>>> and 2) strips the binaries. So the IPA module *.so file installed >>>> isn't the one which the signature was calculated against. Then >>>> the signature check fails, and libcamera tries to run the IPA >>>> module isolated (in a sandbox), which doesn't work if the IPA >>>> module wasn't designed to run isolated. The easiest way to fix that >>>> is to disable splitting out debug information and stripping the binaries >>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >>>> INHIBIT_PACKAGE_STRIP. >>>> >>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >>>> --- >>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >>>> 1 file changed, 8 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> index 00a5c480d..573366f08 100644 >>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >>>> >>>> S = "${WORKDIR}/git" >>>> >>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >>>> >>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >>>> >>>> inherit meson pkgconfig python3native >>>> >>>> +do_install_append() { >>>> + chrpath -d ${D}${libdir}/libcamera.so >>>> +} >>>> + >>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >>>> FILES_${PN} += " ${libdir}/libcamera.so" >>>> >>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >>>> +INHIBIT_PACKAGE_STRIP = "1" >>> >>> I think this is sub-optimal, it means we can not have stripped >>> binaries and it will increase the size unnecessarily >> >> Indeed. >> >> But the alternative is to recalculate the signature on the stripped binary in do_install_append(), >> and the drawback of this is moving part of the module signature implementation into the recipe. >> >> Or the libcamera implementation is to be changed to handle stripped binaries. > > We could calculate the signature on selected sections only, but that > would make the implementation much more complex. Could the > src/ipa/ipa-sign-install.sh resign script could be used by the recipe ? Yes, it could. I'll try this. > We can also improve the script to facilitate its usage. OK. Thanks, Andrey >>>> + >
Hi Khem, On 27.07.2020 18:38, Khem Raj wrote: > On Mon, Jul 27, 2020 at 8:36 AM Andrey Konovalov > <andrey.konovalov@linaro.org> wrote: >> >> Hi Khem, >> >> On 27.07.2020 18:28, Khem Raj wrote: >>> On Mon, Jul 27, 2020 at 2:21 AM Andrey Konovalov >>> <andrey.konovalov@linaro.org> wrote: >>>> >>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in >>>> libcamera.so. If it does, it assumes that libcamera binaries are >>>> run directly from the build directory without installing them, and >>>> tries to use resorces like IPA modules from the build directory. >>>> Mainline meson strips RPATH/RUNPATH out at install time (for >>>> meson versions up to 0.54; the things are somewhat changed in 0.55). >>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >>>> That's why we need to remove this tag manually in do_install_append(). >>>> >>>> IPA module is signed (with openssl dgst) after it is built. But >>>> during packaging the OE build system 1) splits out debugging info, >>>> and 2) strips the binaries. So the IPA module *.so file installed >>>> isn't the one which the signature was calculated against. Then >>>> the signature check fails, and libcamera tries to run the IPA >>>> module isolated (in a sandbox), which doesn't work if the IPA >>>> module wasn't designed to run isolated. The easiest way to fix that >>>> is to disable splitting out debug information and stripping the binaries >>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >>>> INHIBIT_PACKAGE_STRIP. >>>> >>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >>>> --- >>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >>>> 1 file changed, 8 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> index 00a5c480d..573366f08 100644 >>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >>>> >>>> S = "${WORKDIR}/git" >>>> >>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >>>> >>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >>>> >>>> inherit meson pkgconfig python3native >>>> >>>> +do_install_append() { >>>> + chrpath -d ${D}${libdir}/libcamera.so >>>> +} >>>> + >>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >>>> FILES_${PN} += " ${libdir}/libcamera.so" >>>> >>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >>>> +INHIBIT_PACKAGE_STRIP = "1" >>> >>> I think this is sub-optimal, it means we can not have stripped >>> binaries and it will increase the size unnecessarily >> >> Indeed. >> >> But the alternative is to recalculate the signature on the stripped binary in do_install_append(), >> and the drawback of this is moving part of the module signature implementation into the recipe. >> >> Or the libcamera implementation is to be changed to handle stripped binaries. >> > > either of these solutions will be better perhaps. Sound good. I'll try recalculating the signature and post v2 of the patch. Thanks, Andrey
Hi Andrey, On Mon, Jul 27, 2020 at 06:46:47PM +0300, Andrey Konovalov wrote: > On 27.07.2020 14:11, Laurent Pinchart wrote: > > On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: > >> On 27.07.2020 12:42, Kieran Bingham wrote: > >>> On 27/07/2020 10:21, Andrey Konovalov wrote: > >>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >>>> libcamera.so. If it does, it assumes that libcamera binaries are > >>>> run directly from the build directory without installing them, and > >>>> tries to use resorces like IPA modules from the build directory. > >>>> Mainline meson strips RPATH/RUNPATH out at install time (for > >>>> meson versions up to 0.54; the things are somewhat changed in 0.55). > >>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >>>> That's why we need to remove this tag manually in do_install_append(). > >>> > >>> Uh oh, what's changed... (I'll have to go take a look). > >>> > >>> - > >>> https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > >>> > >>> If we're reliant upon meson behaviour which is no longer consistent, > >>> then we are going to have to do something else in libcamera. > >> > >> I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > >> just as before for "usual" (as per libcamera's README) libcamera build. And > >> starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > >> *might* be dropped - if all the packages would be able to set RUNPATH to > >> what they need, and meson would detect that OK in all those cases. > > > > I think that if the problem is caused by a meson patch in openembedded, > > then it would make sense to fix it there. We can decide to address the > > issue in libcamera itself if it's found to affect other distributions > > too, or if meson's behaviour changes in an incompatible way. > > It looks like it is not openembedded only issue: > > -------- Forwarded Message -------- > Subject: [libcamera-devel] [PATCH v4 0/2] package/libcamera: bump version to 96fab38 > Date: Tue, 16 Jun 2020 20:59:49 +0200 > From: Peter Seiderer <ps.report@gmx.net> > To: buildroot@busybox.net > CC: libcamera-devel@lists.libcamera.org, Yann E . MORIN <yann.morin.1998@free.fr> > > <snip> > > With the following patch libcamera is forced to believe it is running > in a installed environment: > > diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp > index d55338f..4ff9dac 100644 > --- a/src/libcamera/utils.cpp > +++ b/src/libcamera/utils.cpp > @@ -346,15 +346,18 @@ details::StringSplitter split(const std::string &str, const std::string &delim) > */ > bool isLibcameraInstalled() > { > +#if 0 > /* > * DT_RUNPATH (DT_RPATH when the linker uses old dtags) is removed on > * install. > */ > for (const ElfW(Dyn) *dyn = _DYNAMIC; dyn->d_tag != DT_NULL; ++dyn) { > - if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) > + if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) { > + printf("XXXXX - dyn->d_un.d_ptr: %s\n", (char*)dyn->d_un.d_ptr); > return false; > + } > } > - > +#endif > return true; > } > > Maybe this is because of the buildroot local meson patch ([1]), leading > to an empty (but not absent) RPATH? buildroot preserves empty RPATH when installing. Maybe we could adapt isLibcameraInstalled() to return true only if RPATH is found *and* not empty ? For openembedded, why is RPATH stripping skipped ? > <snip> > > [0:02:18.125804232] [252] DEBUG IPAManager ipa_manager.cpp:316 IPA module /usr/lib/libcamera/ipa_rpi.so signature is not valid > > <snip> > > This can be avoided with the following patch/hack (disable signature check): > > diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp > index 505cf61..3d64898 100644 > --- a/src/libcamera/ipa_manager.cpp > +++ b/src/libcamera/ipa_manager.cpp > @@ -301,6 +301,9 @@ std::unique_ptr<IPAProxy> IPAManager::createIPA(PipelineHandler *pipe, > > bool IPAManager::isSignatureValid(IPAModule *ipa) const > { > +#if 1 > + return true; > +#else > #if HAVE_IPA_PUBKEY > File file{ ipa->path() }; > if (!file.open(File::ReadOnly)) > @@ -320,6 +323,7 @@ bool IPAManager::isSignatureValid(IPAModule *ipa) const > #else > return false; > #endif > +#endif > } > > } /* namespace libcamera */ > > > Maybe related to the buildroot finalize and/or sanitizing RPATH in target tree > step (and/or strip after install with BR2_ENABLE_DEBUG=y/BR2_STRIP_strip=y > enabled)? For this, let's first see how module re-signing works with openemebedded, possibly improving the resigning script. If it's successful, I think we can then use the same methods for buildroot and other distributions. A packaging document to explain all this would be useful. > -------- End of Forwarded Message -------- > > >>> /me sighs ... > >>> > >>>> IPA module is signed (with openssl dgst) after it is built. But > >>>> during packaging the OE build system 1) splits out debugging info, > >>>> and 2) strips the binaries. So the IPA module *.so file installed > >>>> isn't the one which the signature was calculated against. Then > >>>> the signature check fails, and libcamera tries to run the IPA > >>>> module isolated (in a sandbox), which doesn't work if the IPA > >>>> module wasn't designed to run isolated. The easiest way to fix that > >>>> is to disable splitting out debug information and stripping the binaries > >>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >>>> INHIBIT_PACKAGE_STRIP. > >>> > >>> This sounds like an effective solution for openembedded, but it needs to > >>> be fixed in libcamera all the same. > >>> > >>> I'll try to follow up with the meson guys to see what we can do,. > > > > We re-sign the IPA modules at install time for this very specific > > reason. If openembedded modifies the binaries after installing them, > > should it re-run the signing script ? > > > >>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >>>> --- > >>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >>>> 1 file changed, 8 insertions(+), 1 deletion(-) > >>>> > >>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>> index 00a5c480d..573366f08 100644 > >>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >>>> > >>>> S = "${WORKDIR}/git" > >>>> > >>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >>>> > >>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >>>> > >>>> inherit meson pkgconfig python3native > >>>> > >>>> +do_install_append() { > >>>> + chrpath -d ${D}${libdir}/libcamera.so > >>> > >>> Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > >>> the solution will be handling our own strip/install actions to do this > >>> explicitly in the build. > >>> > >>> It will be a pain to have to pull in another external dependency though... > >>> > >>>> +} > >>>> + > >>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >>>> FILES_${PN} += " ${libdir}/libcamera.so" > >>>> > >>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >>>> +INHIBIT_PACKAGE_STRIP = "1" > >>>> +
Hi Laurent, On 27.07.2020 19:03, Laurent Pinchart wrote: > Hi Andrey, > > On Mon, Jul 27, 2020 at 06:46:47PM +0300, Andrey Konovalov wrote: >> On 27.07.2020 14:11, Laurent Pinchart wrote: >>> On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: >>>> On 27.07.2020 12:42, Kieran Bingham wrote: >>>>> On 27/07/2020 10:21, Andrey Konovalov wrote: >>>>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in >>>>>> libcamera.so. If it does, it assumes that libcamera binaries are >>>>>> run directly from the build directory without installing them, and >>>>>> tries to use resorces like IPA modules from the build directory. >>>>>> Mainline meson strips RPATH/RUNPATH out at install time (for >>>>>> meson versions up to 0.54; the things are somewhat changed in 0.55). >>>>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. >>>>>> That's why we need to remove this tag manually in do_install_append(). >>>>> >>>>> Uh oh, what's changed... (I'll have to go take a look). >>>>> >>>>> - >>>>> https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful >>>>> >>>>> If we're reliant upon meson behaviour which is no longer consistent, >>>>> then we are going to have to do something else in libcamera. >>>> >>>> I haven't tried meson 0.55 yet, but my impression was that 0.55 should work >>>> just as before for "usual" (as per libcamera's README) libcamera build. And >>>> starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal >>>> *might* be dropped - if all the packages would be able to set RUNPATH to >>>> what they need, and meson would detect that OK in all those cases. >>> >>> I think that if the problem is caused by a meson patch in openembedded, >>> then it would make sense to fix it there. We can decide to address the >>> issue in libcamera itself if it's found to affect other distributions >>> too, or if meson's behaviour changes in an incompatible way. >> >> It looks like it is not openembedded only issue: >> >> -------- Forwarded Message -------- >> Subject: [libcamera-devel] [PATCH v4 0/2] package/libcamera: bump version to 96fab38 >> Date: Tue, 16 Jun 2020 20:59:49 +0200 >> From: Peter Seiderer <ps.report@gmx.net> >> To: buildroot@busybox.net >> CC: libcamera-devel@lists.libcamera.org, Yann E . MORIN <yann.morin.1998@free.fr> >> >> <snip> >> >> With the following patch libcamera is forced to believe it is running >> in a installed environment: >> >> diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp >> index d55338f..4ff9dac 100644 >> --- a/src/libcamera/utils.cpp >> +++ b/src/libcamera/utils.cpp >> @@ -346,15 +346,18 @@ details::StringSplitter split(const std::string &str, const std::string &delim) >> */ >> bool isLibcameraInstalled() >> { >> +#if 0 >> /* >> * DT_RUNPATH (DT_RPATH when the linker uses old dtags) is removed on >> * install. >> */ >> for (const ElfW(Dyn) *dyn = _DYNAMIC; dyn->d_tag != DT_NULL; ++dyn) { >> - if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) >> + if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) { >> + printf("XXXXX - dyn->d_un.d_ptr: %s\n", (char*)dyn->d_un.d_ptr); >> return false; >> + } >> } >> - >> +#endif >> return true; >> } >> >> Maybe this is because of the buildroot local meson patch ([1]), leading >> to an empty (but not absent) RPATH? > > buildroot preserves empty RPATH when installing. Maybe we could adapt > isLibcameraInstalled() to return true only if RPATH is found *and* not > empty ? > > For openembedded, why is RPATH stripping skipped ? Not sure if I can give a proper explanation, so I would better refer you to: https://patches.openembedded.org/patch/156584/ https://github.com/mesonbuild/meson/issues/2567 (rpurdie's comments especially) Maybe the fix done in meson 0.55 would let distributions to re-enable RPATH stripping: https://mesonbuild.com/Release-notes-for-0-55-0.html#added-ability-to-specify-targets-in-meson-compile https://github.com/mesonbuild/meson/pull/7103 https://github.com/mesonbuild/meson/pull/7472 - but I don't know if/when this could happen. For me it looks like a lot of re-testing at least (distros have a whole lot of packages which could be affected). >> <snip> >> >> [0:02:18.125804232] [252] DEBUG IPAManager ipa_manager.cpp:316 IPA module /usr/lib/libcamera/ipa_rpi.so signature is not valid >> >> <snip> >> >> This can be avoided with the following patch/hack (disable signature check): >> >> diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp >> index 505cf61..3d64898 100644 >> --- a/src/libcamera/ipa_manager.cpp >> +++ b/src/libcamera/ipa_manager.cpp >> @@ -301,6 +301,9 @@ std::unique_ptr<IPAProxy> IPAManager::createIPA(PipelineHandler *pipe, >> >> bool IPAManager::isSignatureValid(IPAModule *ipa) const >> { >> +#if 1 >> + return true; >> +#else >> #if HAVE_IPA_PUBKEY >> File file{ ipa->path() }; >> if (!file.open(File::ReadOnly)) >> @@ -320,6 +323,7 @@ bool IPAManager::isSignatureValid(IPAModule *ipa) const >> #else >> return false; >> #endif >> +#endif >> } >> >> } /* namespace libcamera */ >> >> >> Maybe related to the buildroot finalize and/or sanitizing RPATH in target tree >> step (and/or strip after install with BR2_ENABLE_DEBUG=y/BR2_STRIP_strip=y >> enabled)? > > For this, let's first see how module re-signing works with > openemebedded, possibly improving the resigning script. If it's > successful, I think we can then use the same methods for buildroot and > other distributions. A packaging document to explain all this would be > useful. Sounds good. Thanks, Andrey >> -------- End of Forwarded Message -------- >> >>>>> /me sighs ... >>>>> >>>>>> IPA module is signed (with openssl dgst) after it is built. But >>>>>> during packaging the OE build system 1) splits out debugging info, >>>>>> and 2) strips the binaries. So the IPA module *.so file installed >>>>>> isn't the one which the signature was calculated against. Then >>>>>> the signature check fails, and libcamera tries to run the IPA >>>>>> module isolated (in a sandbox), which doesn't work if the IPA >>>>>> module wasn't designed to run isolated. The easiest way to fix that >>>>>> is to disable splitting out debug information and stripping the binaries >>>>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and >>>>>> INHIBIT_PACKAGE_STRIP. >>>>> >>>>> This sounds like an effective solution for openembedded, but it needs to >>>>> be fixed in libcamera all the same. >>>>> >>>>> I'll try to follow up with the meson guys to see what we can do,. >>> >>> We re-sign the IPA modules at install time for this very specific >>> reason. If openembedded modifies the binaries after installing them, >>> should it re-run the signing script ? >>> >>>>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> >>>>>> --- >>>>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- >>>>>> 1 file changed, 8 insertions(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>>>> index 00a5c480d..573366f08 100644 >>>>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb >>>>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" >>>>>> >>>>>> S = "${WORKDIR}/git" >>>>>> >>>>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" >>>>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" >>>>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" >>>>>> >>>>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" >>>>>> >>>>>> inherit meson pkgconfig python3native >>>>>> >>>>>> +do_install_append() { >>>>>> + chrpath -d ${D}${libdir}/libcamera.so >>>>> >>>>> Aha, I didn't know about chrpath, that looks helpful. Perhaps part of >>>>> the solution will be handling our own strip/install actions to do this >>>>> explicitly in the build. >>>>> >>>>> It will be a pain to have to pull in another external dependency though... >>>>> >>>>>> +} >>>>>> + >>>>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" >>>>>> FILES_${PN} += " ${libdir}/libcamera.so" >>>>>> >>>>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" >>>>>> +INHIBIT_PACKAGE_STRIP = "1" >>>>>> + >
Hi Andrey, On Mon, Jul 27, 2020 at 07:17:04PM +0300, Andrey Konovalov wrote: > On 27.07.2020 19:03, Laurent Pinchart wrote: > > On Mon, Jul 27, 2020 at 06:46:47PM +0300, Andrey Konovalov wrote: > >> On 27.07.2020 14:11, Laurent Pinchart wrote: > >>> On Mon, Jul 27, 2020 at 12:58:23PM +0300, Andrey Konovalov wrote: > >>>> On 27.07.2020 12:42, Kieran Bingham wrote: > >>>>> On 27/07/2020 10:21, Andrey Konovalov wrote: > >>>>>> libcamera checks if RPATH or RUNPATH dynamic tag is present in > >>>>>> libcamera.so. If it does, it assumes that libcamera binaries are > >>>>>> run directly from the build directory without installing them, and > >>>>>> tries to use resorces like IPA modules from the build directory. > >>>>>> Mainline meson strips RPATH/RUNPATH out at install time (for > >>>>>> meson versions up to 0.54; the things are somewhat changed in 0.55). > >>>>>> But openembedded-core patches meson to disable RPATH/RUNPATH removal. > >>>>>> That's why we need to remove this tag manually in do_install_append(). > >>>>> > >>>>> Uh oh, what's changed... (I'll have to go take a look). > >>>>> > >>>>> - > >>>>> https://mesonbuild.com/Release-notes-for-0-55-0.html#rpath-removal-now-more-careful > >>>>> > >>>>> If we're reliant upon meson behaviour which is no longer consistent, > >>>>> then we are going to have to do something else in libcamera. > >>>> > >>>> I haven't tried meson 0.55 yet, but my impression was that 0.55 should work > >>>> just as before for "usual" (as per libcamera's README) libcamera build. And > >>>> starting from 0.55 the patch in openembedded-core to disable RPATH/RUNPATH removal > >>>> *might* be dropped - if all the packages would be able to set RUNPATH to > >>>> what they need, and meson would detect that OK in all those cases. > >>> > >>> I think that if the problem is caused by a meson patch in openembedded, > >>> then it would make sense to fix it there. We can decide to address the > >>> issue in libcamera itself if it's found to affect other distributions > >>> too, or if meson's behaviour changes in an incompatible way. > >> > >> It looks like it is not openembedded only issue: > >> > >> -------- Forwarded Message -------- > >> Subject: [libcamera-devel] [PATCH v4 0/2] package/libcamera: bump version to 96fab38 > >> Date: Tue, 16 Jun 2020 20:59:49 +0200 > >> From: Peter Seiderer <ps.report@gmx.net> > >> To: buildroot@busybox.net > >> CC: libcamera-devel@lists.libcamera.org, Yann E . MORIN <yann.morin.1998@free.fr> > >> > >> <snip> > >> > >> With the following patch libcamera is forced to believe it is running > >> in a installed environment: > >> > >> diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp > >> index d55338f..4ff9dac 100644 > >> --- a/src/libcamera/utils.cpp > >> +++ b/src/libcamera/utils.cpp > >> @@ -346,15 +346,18 @@ details::StringSplitter split(const std::string &str, const std::string &delim) > >> */ > >> bool isLibcameraInstalled() > >> { > >> +#if 0 > >> /* > >> * DT_RUNPATH (DT_RPATH when the linker uses old dtags) is removed on > >> * install. > >> */ > >> for (const ElfW(Dyn) *dyn = _DYNAMIC; dyn->d_tag != DT_NULL; ++dyn) { > >> - if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) > >> + if (dyn->d_tag == DT_RUNPATH || dyn->d_tag == DT_RPATH) { > >> + printf("XXXXX - dyn->d_un.d_ptr: %s\n", (char*)dyn->d_un.d_ptr); > >> return false; > >> + } > >> } > >> - > >> +#endif > >> return true; > >> } > >> > >> Maybe this is because of the buildroot local meson patch ([1]), leading > >> to an empty (but not absent) RPATH? > > > > buildroot preserves empty RPATH when installing. Maybe we could adapt > > isLibcameraInstalled() to return true only if RPATH is found *and* not > > empty ? > > > > For openembedded, why is RPATH stripping skipped ? > > Not sure if I can give a proper explanation, so I would better refer you to: > > https://patches.openembedded.org/patch/156584/ > https://github.com/mesonbuild/meson/issues/2567 (rpurdie's comments especially) Thank you for the pointers, it's pretty interesting. > Maybe the fix done in meson 0.55 would let distributions to re-enable RPATH stripping: > https://mesonbuild.com/Release-notes-for-0-55-0.html#added-ability-to-specify-targets-in-meson-compile > https://github.com/mesonbuild/meson/pull/7103 > https://github.com/mesonbuild/meson/pull/7472 > - but I don't know if/when this could happen. For me it looks like a lot of re-testing > at least (distros have a whole lot of packages which could be affected). Yes, it will likely take time, even if I think it's the right solution. In the meantime, I think we'll have to handle this in the OE and buildroot recipes, I don't really see what else we could do. It may be possible to improve the implementation in libcamera by checking the R(UN)PATH value (it's tricky though, glibc and musl handle this quite differently, where glibc patches the _DYNAMIC entries with pointers to the string table in memory, while musl doesn't AFAIR), but it would only help if we can get the build system to strip the custom R(UN)PATH entry that we add. If nothing is stripped, I think we're stuck. > >> <snip> > >> > >> [0:02:18.125804232] [252] DEBUG IPAManager ipa_manager.cpp:316 IPA module /usr/lib/libcamera/ipa_rpi.so signature is not valid > >> > >> <snip> > >> > >> This can be avoided with the following patch/hack (disable signature check): > >> > >> diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp > >> index 505cf61..3d64898 100644 > >> --- a/src/libcamera/ipa_manager.cpp > >> +++ b/src/libcamera/ipa_manager.cpp > >> @@ -301,6 +301,9 @@ std::unique_ptr<IPAProxy> IPAManager::createIPA(PipelineHandler *pipe, > >> > >> bool IPAManager::isSignatureValid(IPAModule *ipa) const > >> { > >> +#if 1 > >> + return true; > >> +#else > >> #if HAVE_IPA_PUBKEY > >> File file{ ipa->path() }; > >> if (!file.open(File::ReadOnly)) > >> @@ -320,6 +323,7 @@ bool IPAManager::isSignatureValid(IPAModule *ipa) const > >> #else > >> return false; > >> #endif > >> +#endif > >> } > >> > >> } /* namespace libcamera */ > >> > >> > >> Maybe related to the buildroot finalize and/or sanitizing RPATH in target tree > >> step (and/or strip after install with BR2_ENABLE_DEBUG=y/BR2_STRIP_strip=y > >> enabled)? > > > > For this, let's first see how module re-signing works with > > openemebedded, possibly improving the resigning script. If it's > > successful, I think we can then use the same methods for buildroot and > > other distributions. A packaging document to explain all this would be > > useful. > > Sounds good. > > >> -------- End of Forwarded Message -------- > >> > >>>>> /me sighs ... > >>>>> > >>>>>> IPA module is signed (with openssl dgst) after it is built. But > >>>>>> during packaging the OE build system 1) splits out debugging info, > >>>>>> and 2) strips the binaries. So the IPA module *.so file installed > >>>>>> isn't the one which the signature was calculated against. Then > >>>>>> the signature check fails, and libcamera tries to run the IPA > >>>>>> module isolated (in a sandbox), which doesn't work if the IPA > >>>>>> module wasn't designed to run isolated. The easiest way to fix that > >>>>>> is to disable splitting out debug information and stripping the binaries > >>>>>> during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and > >>>>>> INHIBIT_PACKAGE_STRIP. > >>>>> > >>>>> This sounds like an effective solution for openembedded, but it needs to > >>>>> be fixed in libcamera all the same. > >>>>> > >>>>> I'll try to follow up with the meson guys to see what we can do,. > >>> > >>> We re-sign the IPA modules at install time for this very specific > >>> reason. If openembedded modifies the binaries after installing them, > >>> should it re-run the signing script ? > >>> > >>>>>> Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> > >>>>>> --- > >>>>>> .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- > >>>>>> 1 file changed, 8 insertions(+), 1 deletion(-) > >>>>>> > >>>>>> diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>>>> index 00a5c480d..573366f08 100644 > >>>>>> --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>>>> +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb > >>>>>> @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" > >>>>>> > >>>>>> S = "${WORKDIR}/git" > >>>>>> > >>>>>> -DEPENDS = "python3-pyyaml-native udev gnutls boost" > >>>>>> +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" > >>>>>> DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" > >>>>>> > >>>>>> RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" > >>>>>> > >>>>>> inherit meson pkgconfig python3native > >>>>>> > >>>>>> +do_install_append() { > >>>>>> + chrpath -d ${D}${libdir}/libcamera.so > >>>>> > >>>>> Aha, I didn't know about chrpath, that looks helpful. Perhaps part of > >>>>> the solution will be handling our own strip/install actions to do this > >>>>> explicitly in the build. > >>>>> > >>>>> It will be a pain to have to pull in another external dependency though... > >>>>> > >>>>>> +} > >>>>>> + > >>>>>> FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" > >>>>>> FILES_${PN} += " ${libdir}/libcamera.so" > >>>>>> > >>>>>> +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" > >>>>>> +INHIBIT_PACKAGE_STRIP = "1" > >>>>>> +
diff --git a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb index 00a5c480d..573366f08 100644 --- a/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb +++ b/meta-multimedia/recipes-multimedia/libcamera/libcamera.bb @@ -18,13 +18,20 @@ PV = "202006+git${SRCPV}" S = "${WORKDIR}/git" -DEPENDS = "python3-pyyaml-native udev gnutls boost" +DEPENDS = "python3-pyyaml-native udev gnutls boost chrpath-native" DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'qt', 'qtbase qtbase-native', '', d)}" RDEPENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'wayland qt', 'qtwayland', '', d)}" inherit meson pkgconfig python3native +do_install_append() { + chrpath -d ${D}${libdir}/libcamera.so +} + FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig" FILES_${PN} += " ${libdir}/libcamera.so" +INHIBIT_PACKAGE_DEBUG_SPLIT = "1" +INHIBIT_PACKAGE_STRIP = "1" +
libcamera checks if RPATH or RUNPATH dynamic tag is present in libcamera.so. If it does, it assumes that libcamera binaries are run directly from the build directory without installing them, and tries to use resorces like IPA modules from the build directory. Mainline meson strips RPATH/RUNPATH out at install time (for meson versions up to 0.54; the things are somewhat changed in 0.55). But openembedded-core patches meson to disable RPATH/RUNPATH removal. That's why we need to remove this tag manually in do_install_append(). IPA module is signed (with openssl dgst) after it is built. But during packaging the OE build system 1) splits out debugging info, and 2) strips the binaries. So the IPA module *.so file installed isn't the one which the signature was calculated against. Then the signature check fails, and libcamera tries to run the IPA module isolated (in a sandbox), which doesn't work if the IPA module wasn't designed to run isolated. The easiest way to fix that is to disable splitting out debug information and stripping the binaries during packaging with INHIBIT_PACKAGE_DEBUG_SPLIT and INHIBIT_PACKAGE_STRIP. Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org> --- .../recipes-multimedia/libcamera/libcamera.bb | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)