Re: [PATCH 3/3] vm-img: Handle sparse files efficiently

[Henning Schild <henning.schild@siemens.com>]

Am Mon, 7 Mar 2022 15:41:04 +0100
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 07.03.22 15:09, Henning Schild wrote:
> > Am Mon, 7 Mar 2022 10:46:34 +0100
> > schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >   
> >> On 07.03.22 10:04, Henning Schild wrote:  
> >>> Am Sat, 5 Mar 2022 17:34:50 +0100
> >>> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >>>     
> >>>> From: Jan Kiszka <jan.kiszka@siemens.com>
> >>>>
> >>>> Saves space in case the image contains any sparse files.
> >>>>
> >>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> >>>> ---
> >>>>  meta/classes/vm-img.bbclass | 6 +++---
> >>>>  1 file changed, 3 insertions(+), 3 deletions(-)
> >>>>
> >>>> diff --git a/meta/classes/vm-img.bbclass
> >>>> b/meta/classes/vm-img.bbclass index 4bc977b9..27aedf12 100644
> >>>> --- a/meta/classes/vm-img.bbclass
> >>>> +++ b/meta/classes/vm-img.bbclass
> >>>> @@ -93,15 +93,15 @@ do_create_ova() {
> >>>>          export VM_UUID=$(uuidgen)
> >>>>          # create ovf
> >>>>          cat ${PP_WORK}/${OVF_TEMPLATE_STAGE2} | envsubst >
> >>>> ${PP_DEPLOY}/${OVA_NAME}.ovf
> >>>> -        tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> >>>> ${OVA_NAME}.ovf
> >>>> +        tar -cvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> >>>> ${OVA_NAME}.ovf 
> >>>>          # VirtualBox needs here a manifest file. VMware does
> >>>> accept that format. if [ "${VMDK_SUBFORMAT}" = "monolithicSparse"
> >>>> ]; then    
> >>>
> >>> Not sure that already suggested it but we did not do it.
> >>>     
> >>>>              echo
> >>>> "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum
> >>>> ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >>
> >>>> ${PP_DEPLOY}/${OVA_NAME}.mf echo
> >>>> "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum
> >>>> ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >>
> >>>> ${PP_DEPLOY}/${OVA_NAME}.mf
> >>>> -            tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C
> >>>> ${PP_DEPLOY} ${OVA_NAME}.mf
> >>>> +            tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C
> >>>> ${PP_DEPLOY} ${OVA_NAME}.mf fi
> >>>> -        tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> >>>> ${VIRTUAL_MACHINE_IMAGE_FILE}
> >>>> +        tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> >>>> ${VIRTUAL_MACHINE_IMAGE_FILE} EOSUDO    
> >>>
> >>> Did you get around to testing this on vmware and vbox? Do you
> >>> think it would need to be tested? I could to that.
> >>>     
> >>
> >> That would be valuable. Just add a sparse file to the image (dd
> >> if=/dev/zero of=sparse.file bs=1G seek=1 count=0) and check if that
> >> bloats the files. And, of course, check if things still boot.  
> > 
> > NACK, breaks ova import into both vmware and virtualbox. I compared
> > and it is really this patch which breaks both.
> > 
> > Did not check the vmware logs but vbox says
> > VERR_TAR_UNSUPPORTED_GNU_HDR_TYPE  
> 
> Amazingly primitive format, this OVO...

Not sure, the ova is just a tar of the disk image and an xml. That disk
image itself can itself be sparse.

see VMDK_SUBFORMAT

> > 
> > So this tarball does not support it because its consumers both do
> > not. 
> 
> OK, thanks for testing. We may then consider whether to take patch 2
> as well as it may surprise constrained consumers. Patch 1 is fine as
> it only targets Isar itself.

I think p2 can default to -S and maybe make the args configurable.

But i think Adriaan might have some generic remarks on the targz-img
anyways.

I usually use it for nfs so for me the S would add value. If i remember
to specify it on unpack ;).

Henning
 
> Jan
>