Bug mcopy (wic) when creating bootpartition

Bug mcopy (wic) when creating bootpartition

[benbrenson89]

[-- Attachment #1.1: Type: text/plain, Size: 644 bytes --]

Hi guys,

I have encountered a bug related to mcopy during wic image generation.

I digged a bit deeper into the problem and it turns out that mcopy is not 
able to copy folders recursive. It just silently skips the content of those 
folders.

When running mcopy within the buildchroot-host, these folders where 
properly copied.

My question now:

Wic is running in the buildchroot of the target architecture, but I don't 
get why it has to be done in there?
May it be possible to switch the wic image generation into the 
host-buildchroot, or are there any other requirements for the current 
environment which I miss here?


Greets Benedikt

[-- Attachment #1.2: Type: text/html, Size: 857 bytes --]

Re: Bug mcopy (wic) when creating bootpartition

[Jan Kiszka]

On 27.02.20 10:32, benbrenson89 via isar-users wrote:
> Hi guys,
> 
> I have encountered a bug related to mcopy during wic image generation.
> 
> I digged a bit deeper into the problem and it turns out that mcopy is 
> not able to copy folders recursive. It just silently skips the content 
> of those folders.
> 
> When running mcopy within the buildchroot-host, these folders where 
> properly copied.
> 
> My question now:
> 
> Wic is running in the buildchroot of the target architecture, but I 
> don't get why it has to be done in there?
> May it be possible to switch the wic image generation into the 
> host-buildchroot, or are there any other requirements for the current 
> environment which I miss here?

Running always in the target environment reduces variations because we 
have to do that anyway when cross-compiling is disabled (which is 
default and recommended for production build).

Can you extract the problem to something that can be tested separately? 
What is your target architecture? What is the qemu-user-static version 
you are effectively using?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: Bug mcopy (wic) when creating bootpartition

[benbrenson89]

[-- Attachment #1.1: Type: text/plain, Size: 871 bytes --]

Hi Jan,

Current architecture: armhf
Qemu: qemu-arm-static

To reproduce the bug I jumped into the buildchroot and typed the following:


I tested it with mcopy version 4.0.18 and 4.0.23, with same results.

sudo chroot buildchroot-target-debian-buster-armhf/rootfs
cd /tmp
mkdosfs -n test -S 512 -C test.img $((512*2*1024*100))
mkdir testfolder
touch testfolder/testtestfile1
touch testfolder/testtestfile1
export MTOOLS_SKIP_CHECK=1
mcopy -v -i test.img -s testfolder/ ::/
> Copying testfolder

sudo chroot buildchroot-host-debian-buster-armhf/rootfs
apt-get install mtools dosfstools
cd /tmp
mkdosfs -n test -S 512 -C test.img $((512*2*1024*100))
mkdir testfolder
touch testfolder/testtestfile1
touch testfolder/testtestfile1
export MTOOLS_SKIP_CHECK=1
mcopy -v -i test.img -s testfolder/ ::/
> Copying testfolder
> Copying testfile2
> Copying testfile1


Greets



 

[-- Attachment #1.2: Type: text/html, Size: 1335 bytes --]

Re: Bug mcopy (wic) when creating bootpartition

[Jan Kiszka]

On 27.02.20 13:33, benbrenson89 via isar-users wrote:
> Hi Jan,
> 
> Current architecture: armhf
> Qemu: qemu-arm-static
> 
> To reproduce the bug I jumped into the buildchroot and typed the following:
> 
> 
> I tested it with mcopy version 4.0.18 and 4.0.23, with same results.
> 
> sudo chroot buildchroot-target-debian-buster-armhf/rootfs
> cd /tmp
> mkdosfs -n test -S 512 -C test.img $((512*2*1024*100))
> mkdir testfolder
> touch testfolder/testtestfile1
> touch testfolder/testtestfile1
> export MTOOLS_SKIP_CHECK=1
> mcopy -v -i test.img -s testfolder/ ::/
>  > Copying testfolder

If you chroot into that rootfs on an ARM target (I suppose QEMU would be 
fine as well), does it then work? That would be the smoking gun for 
qemu-user. If so, using qemu-user from testing might be worth a try.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: Bug mcopy (wic) when creating bootpartition

[Henning Schild]

On Thu, 27 Feb 2020 01:32:31 -0800
benbrenson89 via isar-users <isar-users@googlegroups.com> wrote:

> Hi guys,
> 
> I have encountered a bug related to mcopy during wic image generation.
> 
> I digged a bit deeper into the problem and it turns out that mcopy is
> not able to copy folders recursive. It just silently skips the
> content of those folders.
> 
> When running mcopy within the buildchroot-host, these folders where 
> properly copied.
> 
> My question now:
> 
> Wic is running in the buildchroot of the target architecture, but I
> don't get why it has to be done in there?

Back in the early days of wic we ran it on the host. But that had
really nasty side-effects that are simply not acceptable. It needs to
run in the "same debian" that you want to build for. i.e. so you get
the bootloader binaries of the bootloader you expect

> May it be possible to switch the wic image generation into the 
> host-buildchroot, or are there any other requirements for the current 
> environment which I miss here?

A host-buildchroot would probably work as well. But i doubt it would
solve your problem. But it can not be the default since cross is an
experimental feature that we do not suggest for production. But it
works well to speed up dev builds.

Henning

> 
> Greets Benedikt
>