edk2 fails with isar 0.10

edk2 fails with isar 0.10

[Schultschik, Sven]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

Hi everyone,

we updated https://github.com/ilbers/isar
in our setup from 

From 0.9
213b8f62fa74e87321f927fb6b4ed9db93e5cb37
to 0.10 dev
51f5b0dbecb4a7271540793e13c54fac69b42da0


With this update edk2 standalonemm rpmb build started to faile because
of 

/usr/lib/gcc-cross/aarch64-linux-gnu/10/../../../../aarch64-linux-
gnu/bin/ld: cannot find -llto-aarch64
 
the same error, if we deactivate cross build

if we go back to 0.9 the error does not apear

It doesn't matter if we are using edk2 202302 oder 202311 version of
edk2.

we normaly build with GCC5 toolchain flag (GCC for 202311 an newer)

If we switch to edk2 toolchain GCC49 (GCCNOLTO for 202311 and newer) it
compiles without any further errors.

So something in that update to 0.10 seems to prevent LTO to be not
available anymore.

Did someone see similar already, or already knows the reason?

br Sven 


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 10057 bytes --]

Re: edk2 fails with isar 0.10

[Anton Mikanovich]

31/01/2024 12:09, 'Schultschik, Sven' via isar-users wrote:
> Hi everyone,
>
> we updated https://github.com/ilbers/isar
> in our setup from
>
>  From 0.9
> 213b8f62fa74e87321f927fb6b4ed9db93e5cb37
> to 0.10 dev
> 51f5b0dbecb4a7271540793e13c54fac69b42da0
>
>
> With this update edk2 standalonemm rpmb build started to faile because
> of
>
> /usr/lib/gcc-cross/aarch64-linux-gnu/10/../../../../aarch64-linux-
> gnu/bin/ld: cannot find -llto-aarch64
>   
> the same error, if we deactivate cross build
>
> if we go back to 0.9 the error does not apear
>
> It doesn't matter if we are using edk2 202302 oder 202311 version of
> edk2.
>
> we normaly build with GCC5 toolchain flag (GCC for 202311 an newer)
>
> If we switch to edk2 toolchain GCC49 (GCCNOLTO for 202311 and newer) it
> compiles without any further errors.
>
> So something in that update to 0.10 seems to prevent LTO to be not
> available anymore.
>
> Did someone see similar already, or already knows the reason?
>
> br Sven
>
Hello Sven,

Is there any steps to reproduce this issue? (with recipe or layer to try).
I've tried edk2 with GCC5 option outside Isar and it just worked, so we 
need to
check if it's some kind of Isar issue or recipe itself.

The main difference between 0.9 and 0.10rc1 is sbuild integration which 
makes
all recipes been built independently. It requires more strict dependencies
handling, because packages will not be shared between recipes.

Re: edk2 fails with isar 0.10

[Schultschik, Sven]

[-- Attachment #1.1: Type: text/plain, Size: 1008 bytes --]

On Wed, 2024-01-31 at 13:21 +0200, Anton Mikanovich wrote:
> 31/01/2024 12:09, 'Schultschik, Sven' via isar-users wrote:
> Hello Sven,
> 
> Is there any steps to reproduce this issue? (with recipe or layer to
> try).

I attached the recipe and rules file. Hope that's ok.

> I've tried edk2 with GCC5 option outside Isar and it just worked, so
> we 
> need to
> check if it's some kind of Isar issue or recipe itself.

Yes outside of ISAR it just works as expected

> 
> The main difference between 0.9 and 0.10rc1 is sbuild integration
> which 
> makes
> all recipes been built independently. It requires more strict
> dependencies
> handling, because packages will not be shared between recipes.

That was my first thought, but the LTO lib is no extra package so far.
It comes with the gcc compiler. 
When I search for hints I saw that buildroot had changes regarding
support for lto, but I didn't followed the versions and details if it
really matters for this issue yet.



[-- Attachment #1.2: rules.tmpl --]
[-- Type: text/x-makefile, Size: 2193 bytes --]

#!/usr/bin/make -f
#
# Copyright (c) Siemens AG, 2022-2023
#
# Authors:
#  Sven Schultschik <sven.schultschik@siemens.com>
#
# SPDX-License-Identifier: MIT

ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)-
endif

export WORKSPACE=$(shell pwd)
export PACKAGES_PATH=$(WORKSPACE)/edk2:$(WORKSPACE)/edk2-platforms
export ACTIVE_PLATFORM="Platform/StandaloneMm/PlatformStandaloneMmPkg/PlatformStandaloneMmRpmb.dsc"

# https://github.com/tianocore/edk2-platforms/blob/master/Readme.md#if-cross-compiling
ifeq (arm64,$(DEB_TARGET_ARCH))
export TARGET_ARCH = 'AARCH64'
else ifeq ((armhf,$(DEB_TARGET_ARCH))
export TARGET_ARCH = 'ARM'
else ifeq ((amd64,$(DEB_TARGET_ARCH))
export TARGET_ARCH = 'X64'
else ifeq ((i386,$(DEB_TARGET_ARCH))
export TARGET_ARCH = 'IA32'
else
$(error DEB_TARGET_ARCH $(DEB_TARGET_ARCH) unsupported)
endif

export SHELL=/bin/bash

# ENV Vars which should get set by edksetup.sh
export PYTHON_COMMAND=python3
export PYTHONHASHSEED=1
export CONF_PATH=$(WORKSPACE)/edk2/Conf
export EDK_TOOLS_PATH=$(WORKSPACE)/edk2/BaseTools
export PATH=$(WORKSPACE)/edk2/BaseTools/Bin/Linux-$(TARGET_ARCH):$(WORKSPACE)/edk2/BaseTools/BinWrappers/PosixLike::/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# When cross-compiling, or building with a different version of the compiler than
# the default `gcc`, we additionally need to inform the
# build command which toolchain to use. We do this by setting the environment
# variable `{TOOL_CHAIN_TAG}_{TARGET_ARCH}_PREFIX` - in the case above,
# **GCC5_AARCH64_PREFIX**.
# export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
# using export here at TOP Level does not work, because
# GCC5_$(TARGET_ARCH)_PREFIX gets deleted again for what reason ever
# Therefore it is set right before the build command
# export GCC5_$(TARGET_ARCH)_PREFIX=$(DEB_HOST_GNU_TYPE)-

override_dh_auto_build:
	source edk2/edksetup.sh --reconfig

	CFLAGS= LDFLAGS=  make -C edk2/BaseTools

	(export GCC49_$(TARGET_ARCH)_PREFIX=$(DEB_HOST_GNU_TYPE)- && \
		build -p $(ACTIVE_PLATFORM) -b RELEASE -a $(TARGET_ARCH) -t GCC49 -n $(shell nproc))

override_dh_auto_install:

override_dh_auto_test:

%:
	dh $@ --no-parallel

[-- Attachment #1.3: edk2-standalonemm-rpmb-upstream.bb --]
[-- Type: text/plain, Size: 2165 bytes --]

#
# CIP Core, generic profile
#
# Copyright (c) Siemens AG, 2022-2023
#
# Authors:
#  Sven Schultschik <sven.schultschik@siemens.com>
#  Jan Kiszka <jan.kiszka@siemens.com>
#
# SPDX-License-Identifier: MIT
#

HOMEPAGE = "https://github.com/tianocore/edk2"
MAINTAINER = "Sven Schultschik <sven.schultschik@siemens.com>"

inherit dpkg

SRC_URI = " \
    git://github.com/tianocore/edk2.git;protocol=https;branch=${SRC_BRANCH_EDK2};name=edk2;destsuffix=${S}/edk2 \
    git://github.com/tianocore/edk2-platforms.git;protocol=https;branch=${SRC_BRANCH_EDK2_PLAT};name=edk2-platforms;destsuffix=${S}/edk2-platforms \
    git://github.com/google/brotli;protocol=https;branch=${SRC_BRANCH_BROTLI};name=brotli;destsuffix=${S}/brotli \
    git://github.com/openssl/openssl;protocol=https;branch=${SRC_BRANCH_OPENSSL};name=openssl;destsuffix=${S}/openssl \
    file://rules.tmpl \
    "

PV = "202302"

SRC_BRANCH_EDK2 = "master"
#SRCREV_edk2 = " edk2-stable202302"
SRCREV_edk2 = "f80f052277c88a67c55e107b550f504eeea947d3"

SRC_BRANCH_EDK2_PLAT = "master"
# revision closest to edk2 release
SRCREV_edk2-platforms = "4ad557e494d8055f5ea16009d6e565cace6571d6"

SRC_BRANCH_BROTLI = "master"
# according to edk2 submodules
SRCREV_brotli = "f4153a09f87cbb9c826d8fc12c74642bb2d879ea"

SRC_BRANCH_OPENSSL = "OpenSSL_1_1_1-stable"
#SRCREV_openssl = "OpenSSL_1_1_1s"
SRCREV_openssl = "129058165d195e43a0ad10111b0c2e29bdf65980"

S = "${WORKDIR}/git"

DEBIAN_BUILD_DEPENDS = "bash, python3:native, dh-python, uuid-dev:native"

TEMPLATE_FILES = "rules.tmpl"


do_prepare_build() {
    deb_debianize

    rm -rf ${S}/edk2/BaseTools/Source/C/BrotliCompress/brotli
    ln -s ../../../../../brotli ${S}/edk2/BaseTools/Source/C/BrotliCompress/brotli

    rm -rf ${S}/edk2/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
    ln -s ../../../../brotli ${S}/edk2/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli

    rm -rf ${S}/edk2/CryptoPkg/Library/OpensslLib/openssl
    ln -s ../../../../openssl ${S}/edk2/CryptoPkg/Library/OpensslLib/openssl

    echo "Build/MmStandaloneRpmb/RELEASE_GCC49/FV/BL32_AP_MM.fd /usr/lib/edk2/" > \
        ${S}/debian/edk2-standalonemm-rpmb.install
}

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 10057 bytes --]

Re: edk2 fails with isar 0.10

[Jan Kiszka]

On 31.01.24 14:07, 'Schultschik, Sven' via isar-users wrote:
> On Wed, 2024-01-31 at 13:21 +0200, Anton Mikanovich wrote:
>> 31/01/2024 12:09, 'Schultschik, Sven' via isar-users wrote:
>> Hello Sven,
>>
>> Is there any steps to reproduce this issue? (with recipe or layer to
>> try).
> 
> I attached the recipe and rules file. Hope that's ok.
> 
>> I've tried edk2 with GCC5 option outside Isar and it just worked, so
>> we 
>> need to
>> check if it's some kind of Isar issue or recipe itself.
> 
> Yes outside of ISAR it just works as expected
> 
>>
>> The main difference between 0.9 and 0.10rc1 is sbuild integration
>> which 
>> makes
>> all recipes been built independently. It requires more strict
>> dependencies
>> handling, because packages will not be shared between recipes.
> 
> That was my first thought, but the LTO lib is no extra package so far.
> It comes with the gcc compiler. 
> When I search for hints I saw that buildroot had changes regarding
> support for lto, but I didn't followed the versions and details if it
> really matters for this issue yet.
> 

We are on sbuild Isar in meta-iot2050 (currently
30471bb3f787fb0aa822d15e141ecce2f3b99860), and there the recipe builds.
Already compared that?

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: edk2 fails with isar 0.10

[Schultschik, Sven]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]

On Wed, 2024-01-31 at 16:50 +0100, Jan Kiszka wrote:
> 
> We are on sbuild Isar in meta-iot2050 (currently
> 30471bb3f787fb0aa822d15e141ecce2f3b99860), and there the recipe
> builds.
> Already compared that?
> 
> Jan
> 

Hi Jan,

that's weired. the iot2025 does not have that issue and both recipes
are identical, therefor I wrote both of them ^^

No clue right know where the difference is. Three big ??? right now
over my head

Sven

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 10057 bytes --]

Re: edk2 fails with isar 0.10

['Schultschik, Sven' via isar-users]

Hi Jan,

we are back on that topic with GCC5 within ISAR instead of GCC49.

We are now at the point that the file

./edk2/BaseTools/Bin/GccLto/liblto-aarch64.a

is available in the repository, but seems to get deleted before going
into dh_auto_build

| find -iname *to-aarch64*
| ./edk2/BaseTools/Bin/GccLto/liblto-aarch64.s

So between prepare build and doing the build the .a files seems to get
deleted. But we couldn't yet find where.

The files is available in the recepie git folder.

Sven

On Thu, 2024-02-01 at 16:35 +0000, 'Schultschik, Sven' via isar-users
wrote:
> On Wed, 2024-01-31 at 16:50 +0100, Jan Kiszka wrote:
> > 
> > We are on sbuild Isar in meta-iot2050 (currently
> > 30471bb3f787fb0aa822d15e141ecce2f3b99860), and there the recipe
> > builds.
> > Already compared that?
> > 
> > Jan
> > 
> 
> Hi Jan,
> 
> that's weired. the iot2025 does not have that issue and both recipes
> are identical, therefor I wrote both of them ^^
> 
> No clue right know where the difference is. Three big ??? right now
> over my head
> 
> Sven
> 

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/b72c98574790394aaf48d236e20dd4abecf778a5.camel%40siemens.com.

Re: edk2 fails with isar 0.10

[Baurzhan Ismagulov]

Hello Sven,

On 2024-09-19 16:26, Schultschik, Sven wrote:
> We are now at the point that the file
> 
> ./edk2/BaseTools/Bin/GccLto/liblto-aarch64.a
> 
> is available in the repository, but seems to get deleted before going
> into dh_auto_build
...
> So between prepare build and doing the build the .a files seems to get
> deleted. But we couldn't yet find where.

Possibly related in any way to 30471bb3 Allow disabling source packages
cleanup?

https://groups.google.com/g/isar-users/c/sbrBaLKTncM/m/VG0xeCvcBQAJ

With kind regards,
Baurzhan

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/ZuxsL4SSHNCWIo04%40abai.m.ilbers.de.

Re: edk2 fails with isar 0.10

[Uladzimir Bely]

On Thu, 2024-09-19 at 16:26 +0000, 'Schultschik, Sven' via isar-users
wrote:
> Hi Jan,
> 
> we are back on that topic with GCC5 within ISAR instead of GCC49.
> 
> We are now at the point that the file
> 
> ./edk2/BaseTools/Bin/GccLto/liblto-aarch64.a
> 
> is available in the repository, but seems to get deleted before going
> into dh_auto_build
> 
> > find -iname *to-aarch64*
> > ./edk2/BaseTools/Bin/GccLto/liblto-aarch64.s
> 
> So between prepare build and doing the build the .a files seems to
> get
> deleted. But we couldn't yet find where.
> 
> The files is available in the recepie git folder.
> 
> Sven
> 

Hello Sven.

I remember something similar (e.g., removing original binaries) on a
different project. Newer debhelper was removing any binaries in source
code that had different from target's architecture.

AFAIR, overriding dh_strip (or some different step... dh_shlibdeps?
dh_fixperms?) helped.

```
override_dh_strip:
	@echo Skipping $@
```

BTW, the recipe you attached few messages above, builds well with
recent isar/next (arm64 cross/compile).


> On Thu, 2024-02-01 at 16:35 +0000, 'Schultschik, Sven' via isar-users
> wrote:
> > On Wed, 2024-01-31 at 16:50 +0100, Jan Kiszka wrote:
> > > 
> > > We are on sbuild Isar in meta-iot2050 (currently
> > > 30471bb3f787fb0aa822d15e141ecce2f3b99860), and there the recipe
> > > builds.
> > > Already compared that?
> > > 
> > > Jan
> > > 
> > 
> > Hi Jan,
> > 
> > that's weired. the iot2025 does not have that issue and both
> > recipes
> > are identical, therefor I wrote both of them ^^
> > 
> > No clue right know where the difference is. Three big ??? right now
> > over my head
> > 
> > Sven
> > 
> 

-- 
Best regards,
Uladzimir.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/3fe1efb927180d7890d49c0e02cd2e0d32c018bc.camel%40ilbers.de.

Re: edk2 fails with isar 0.10

['Schultschik, Sven' via isar-users]

[-- Attachment #1: Type: text/plain, Size: 1128 bytes --]

Hi Baurzhan,

thank you for the hint, that was the missing piece.

DPKG_SOURCE_EXTRA_ARGS = "-I!*.a"

Now we have edk2-stable202408 with GCC5 in ISAR running :)

br
Sven

On Thu, 2024-09-19 at 20:23 +0200, Baurzhan Ismagulov wrote:
> Hello Sven,
> 
> On 2024-09-19 16:26, Schultschik, Sven wrote:
> > We are now at the point that the file
> > 
> > ./edk2/BaseTools/Bin/GccLto/liblto-aarch64.a
> > 
> > is available in the repository, but seems to get deleted before
> > going
> > into dh_auto_build
> ...
> > So between prepare build and doing the build the .a files seems to
> > get
> > deleted. But we couldn't yet find where.
> 
> Possibly related in any way to 30471bb3 Allow disabling source
> packages
> cleanup?
> 
> With kind regards,
> Baurzhan
> 

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/ffc368f50e44771acf32953466cad2ff82030815.camel%40siemens.com.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 11786 bytes --]