[PATCH] targz-img: Ignore tar exit code 1

[PATCH] targz-img: Ignore tar exit code 1

[vijaikumar.kanagarajan]

From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>

Sometimes during packaging tar prints a warning "file changed as we read it"
and exits with return code 1. This is observed when we generete both
wic-img and targz-img together. Ignore the error.

References:
https://lists.gnu.org/archive/html/bug-tar/2010-09/msg00017.html
https://github.com/openembedded/openembedded-core/commit/402b4cffbb4c58cfee93db18192f2b218ee0ae35

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
---
 meta/classes/targz-img.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
index d9dadae..68cf80e 100644
--- a/meta/classes/targz-img.bbclass
+++ b/meta/classes/targz-img.bbclass
@@ -7,7 +7,9 @@ TARGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz"
 
 do_targz_image() {
     rm -f ${TARGZ_IMAGE_FILE}
-    sudo tar -cvzf ${TARGZ_IMAGE_FILE} --one-file-system -C ${IMAGE_ROOTFS} .
+    sudo -s <<'EOSUDO'
+        tar -cvzf ${TARGZ_IMAGE_FILE} --one-file-system -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]
+EOSUDO
 }
 
 addtask targz_image before do_image after do_image_tools
-- 
2.17.1

Re: [PATCH] targz-img: Ignore tar exit code 1

[Jan Kiszka]

On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> 
> Sometimes during packaging tar prints a warning "file changed as we read it"
> and exits with return code 1. This is observed when we generete both
> wic-img and targz-img together. Ignore the error.

Ignoring errors is rarely a good idea. In this case, it tells you that
you have a bug in your layer that missing a dependency. I know this
because we once had the same issue in a product layer. So: NAK

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] targz-img: Ignore tar exit code 1

[Vijai Kumar K]

On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:
> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
> > From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > 
> > Sometimes during packaging tar prints a warning "file changed as we read it"
> > and exits with return code 1. This is observed when we generete both
> > wic-img and targz-img together. Ignore the error.
> 
> Ignoring errors is rarely a good idea. In this case, it tells you that
> you have a bug in your layer that missing a dependency. I know this
> because we once had the same issue in a product layer. So: NAK
> 
> Jan

Ah! Ok. Back to square one I guess. Its currently observed in our downstream setups
only. I have no good reason to justify it. Reproducing this issue is gonna be difficult.
Let me see if I can monitor file changes and pinpoint the exact condition.
But this is going to take a while.

If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img is
inherited so both are executed for a build. Also, on a seperate note 
IMAGE_TYPE's functionality could be extended to provide multiple image types.
I believe we could specify only one right now. Former is more clean and particularly
useful if one wants a wic for sd boot and tar for nfsroot. Planning to take this up
next.

Thanks,
Vijai Kumar K

> 
> -- 
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux

Re: [PATCH] targz-img: Ignore tar exit code 1

[Jan Kiszka]

On 05.12.19 20:17, Vijai Kumar K wrote:
> On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:
>> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
>>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
>>>
>>> Sometimes during packaging tar prints a warning "file changed as we read it"
>>> and exits with return code 1. This is observed when we generete both
>>> wic-img and targz-img together. Ignore the error.
>>
>> Ignoring errors is rarely a good idea. In this case, it tells you that
>> you have a bug in your layer that missing a dependency. I know this
>> because we once had the same issue in a product layer. So: NAK
>>
>> Jan
> 
> Ah! Ok. Back to square one I guess. Its currently observed in our downstream setups
> only. I have no good reason to justify it. Reproducing this issue is gonna be difficult.
> Let me see if I can monitor file changes and pinpoint the exact condition.
> But this is going to take a while.
> 
> If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img is
> inherited so both are executed for a build. Also, on a seperate note

wic may have found a reason to modify the rootfs while integrating it.
So a first shot could be making targz depend on wic completion.

BTW, this sounds similar to
https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass
- and that class may have the same issue...

> IMAGE_TYPE's functionality could be extended to provide multiple image types.
> I believe we could specify only one right now. Former is more clean and particularly
> useful if one wants a wic for sd boot and tar for nfsroot. Planning to take this up
> next.

Right, that is of general interest, indeed. Quirin has something planned
around the same use case as well.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] targz-img: Ignore tar exit code 1

[Henning Schild]

Am Thu, 5 Dec 2019 20:31:48 +0100
schrieb "[ext] Jan Kiszka" <jan.kiszka@siemens.com>:

> On 05.12.19 20:17, Vijai Kumar K wrote:
> > On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:  
> >> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:  
> >>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> >>>
> >>> Sometimes during packaging tar prints a warning "file changed as
> >>> we read it" and exits with return code 1. This is observed when
> >>> we generete both wic-img and targz-img together. Ignore the
> >>> error.  
> >>
> >> Ignoring errors is rarely a good idea. In this case, it tells you
> >> that you have a bug in your layer that missing a dependency. I
> >> know this because we once had the same issue in a product layer.
> >> So: NAK
> >>
> >> Jan  
> > 
> > Ah! Ok. Back to square one I guess. Its currently observed in our
> > downstream setups only. I have no good reason to justify it.
> > Reproducing this issue is gonna be difficult. Let me see if I can
> > monitor file changes and pinpoint the exact condition. But this is
> > going to take a while.
> > 
> > If it helps, the scenario is IMAGE_TYPE set to wic-img and
> > targz-img is inherited so both are executed for a build. Also, on a
> > seperate note  
> 
> wic may have found a reason to modify the rootfs while integrating it.
> So a first shot could be making targz depend on wic completion.

I hope it did not. But my first guess would be /etc/fstab. Whichever
file it touched, it should only affect the wic image! Worth checking
whether wic changes the original and maybe fix that.

Henning

> BTW, this sounds similar to
> https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass
> - and that class may have the same issue...
> 
> > IMAGE_TYPE's functionality could be extended to provide multiple
> > image types. I believe we could specify only one right now. Former
> > is more clean and particularly useful if one wants a wic for sd
> > boot and tar for nfsroot. Planning to take this up next.  
> 
> Right, that is of general interest, indeed. Quirin has something
> planned around the same use case as well.
> 
> Jan
> 

Re: [PATCH] targz-img: Ignore tar exit code 1

[Gylstorff Quirin]

On 12/5/19 8:31 PM, Jan Kiszka wrote:
> On 05.12.19 20:17, Vijai Kumar K wrote:
>> On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:
>>> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
>>>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
>>>>
>>>> Sometimes during packaging tar prints a warning "file changed as we read it"
>>>> and exits with return code 1. This is observed when we generete both
>>>> wic-img and targz-img together. Ignore the error.
>>>
>>> Ignoring errors is rarely a good idea. In this case, it tells you that
>>> you have a bug in your layer that missing a dependency. I know this
>>> because we once had the same issue in a product layer. So: NAK
>>>
>>> Jan
>>
>> Ah! Ok. Back to square one I guess. Its currently observed in our downstream setups
>> only. I have no good reason to justify it. Reproducing this issue is gonna be difficult.
>> Let me see if I can monitor file changes and pinpoint the exact condition.
>> But this is going to take a while.
>>
>> If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img is
>> inherited so both are executed for a build. Also, on a seperate note
> 
> wic may have found a reason to modify the rootfs while integrating it.
> So a first shot could be making targz depend on wic completion.
> 
> BTW, this sounds similar to
> https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass
> - and that class may have the same issue...
> 

In case of this class the wic is create first and tarball afterwards. 
 From my expirence I never had the tarball creation failing during the 
cip or xenomai-builds which both use the targz class. Does the error 
during a clean build from scratch or only during a partial rebuild?

>> IMAGE_TYPE's functionality could be extended to provide multiple image types.
>> I believe we could specify only one right now. Former is more clean and particularly
>> useful if one wants a wic for sd boot and tar for nfsroot. Planning to take this up
>> next.
> 
> Right, that is of general interest, indeed. Quirin has something planned
> around the same use case as well.

> 
> Jan
> 
Quirin

Re: [PATCH] targz-img: Ignore tar exit code 1

[vijai kumar]

[-- Attachment #1: Type: text/plain, Size: 2413 bytes --]

Hi Quirin,

Its a scratch build.

Thanks,
Vijai Kumar K

On Fri 6 Dec, 2019, 4:43 PM Gylstorff Quirin, <quirin.gylstorff@siemens.com>
wrote:

>
>
> On 12/5/19 8:31 PM, Jan Kiszka wrote:
> > On 05.12.19 20:17, Vijai Kumar K wrote:
> >> On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:
> >>> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
> >>>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> >>>>
> >>>> Sometimes during packaging tar prints a warning "file changed as we
> read it"
> >>>> and exits with return code 1. This is observed when we generete both
> >>>> wic-img and targz-img together. Ignore the error.
> >>>
> >>> Ignoring errors is rarely a good idea. In this case, it tells you that
> >>> you have a bug in your layer that missing a dependency. I know this
> >>> because we once had the same issue in a product layer. So: NAK
> >>>
> >>> Jan
> >>
> >> Ah! Ok. Back to square one I guess. Its currently observed in our
> downstream setups
> >> only. I have no good reason to justify it. Reproducing this issue is
> gonna be difficult.
> >> Let me see if I can monitor file changes and pinpoint the exact
> condition.
> >> But this is going to take a while.
> >>
> >> If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img is
> >> inherited so both are executed for a build. Also, on a seperate note
> >
> > wic may have found a reason to modify the rootfs while integrating it.
> > So a first shot could be making targz depend on wic completion.
> >
> > BTW, this sounds similar to
> >
> https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass
> > - and that class may have the same issue...
> >
>
> In case of this class the wic is create first and tarball afterwards.
>  From my expirence I never had the tarball creation failing during the
> cip or xenomai-builds which both use the targz class. Does the error
> during a clean build from scratch or only during a partial rebuild?
>
> >> IMAGE_TYPE's functionality could be extended to provide multiple image
> types.
> >> I believe we could specify only one right now. Former is more clean and
> particularly
> >> useful if one wants a wic for sd boot and tar for nfsroot. Planning to
> take this up
> >> next.
> >
> > Right, that is of general interest, indeed. Quirin has something planned
> > around the same use case as well.
>
> >
> > Jan
> >
> Quirin
>

[-- Attachment #2: Type: text/html, Size: 3539 bytes --]

Re: [PATCH] targz-img: Ignore tar exit code 1

[Vijai Kumar K]

On Thu, Dec 05, 2019 at 08:31:48PM +0100, Jan Kiszka wrote:
> On 05.12.19 20:17, Vijai Kumar K wrote:
> > On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote:
> >> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote:
> >>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> >>>
> >>> Sometimes during packaging tar prints a warning "file changed as we read it"
> >>> and exits with return code 1. This is observed when we generete both
> >>> wic-img and targz-img together. Ignore the error.
> >>
> >> Ignoring errors is rarely a good idea. In this case, it tells you that
> >> you have a bug in your layer that missing a dependency. I know this
> >> because we once had the same issue in a product layer. So: NAK
> >>
> >> Jan
> > 
> > Ah! Ok. Back to square one I guess. Its currently observed in our downstream setups
> > only. I have no good reason to justify it. Reproducing this issue is gonna be difficult.
> > Let me see if I can monitor file changes and pinpoint the exact condition.
> > But this is going to take a while.
> > 
> > If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img is
> > inherited so both are executed for a build. Also, on a seperate note
> 
> wic may have found a reason to modify the rootfs while integrating it.
> So a first shot could be making targz depend on wic completion.
> 
> BTW, this sounds similar to
> https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass
> - and that class may have the same issue...
> 
> > IMAGE_TYPE's functionality could be extended to provide multiple image types.
> > I believe we could specify only one right now. Former is more clean and particularly
> > useful if one wants a wic for sd boot and tar for nfsroot. Planning to take this up
> > next.
> 
> Right, that is of general interest, indeed. Quirin has something planned
> around the same use case as well.

Hi Quirin,

Do you have anything planned on this? If not, I can have a look.
Please do let me know.

Thanks,
Vijai Kumar K

> 
> Jan
> 
> -- 
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux

Re: [PATCH] targz-img: Ignore tar exit code 1

[vijai kumar]

[-- Attachment #1.1: Type: text/plain, Size: 2431 bytes --]



On Wednesday, January 29, 2020 at 12:03:56 PM UTC+5:30, vijai kumar wrote:
>
> On Thu, Dec 05, 2019 at 08:31:48PM +0100, Jan Kiszka wrote: 
> > On 05.12.19 20:17, Vijai Kumar K wrote: 
> > > On Thu, Dec 05, 2019 at 07:04:07PM +0100, Jan Kiszka wrote: 
> > >> On 05.12.19 18:44, vijaikumar.kanagarajan@gmail.com wrote: 
> > >>> From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> 
> > >>> 
> > >>> Sometimes during packaging tar prints a warning "file changed as we 
> read it" 
> > >>> and exits with return code 1. This is observed when we generete both 
> > >>> wic-img and targz-img together. Ignore the error. 
> > >> 
> > >> Ignoring errors is rarely a good idea. In this case, it tells you 
> that 
> > >> you have a bug in your layer that missing a dependency. I know this 
> > >> because we once had the same issue in a product layer. So: NAK 
> > >> 
> > >> Jan 
> > > 
> > > Ah! Ok. Back to square one I guess. Its currently observed in our 
> downstream setups 
> > > only. I have no good reason to justify it. Reproducing this issue is 
> gonna be difficult. 
> > > Let me see if I can monitor file changes and pinpoint the exact 
> condition. 
> > > But this is going to take a while. 
> > > 
> > > If it helps, the scenario is IMAGE_TYPE set to wic-img and targz-img 
> is 
> > > inherited so both are executed for a build. Also, on a seperate note 
> > 
> > wic may have found a reason to modify the rootfs while integrating it. 
> > So a first shot could be making targz depend on wic completion. 
> > 
> > BTW, this sounds similar to 
> > 
> https://gitlab.com/cip-project/cip-core/isar-cip-core/blob/next/classes/wic-targz-img.bbclass 
> > - and that class may have the same issue... 
> > 
> > > IMAGE_TYPE's functionality could be extended to provide multiple image 
> types. 
> > > I believe we could specify only one right now. Former is more clean 
> and particularly 
> > > useful if one wants a wic for sd boot and tar for nfsroot. Planning to 
> take this up 
> > > next. 
> > 
> > Right, that is of general interest, indeed. Quirin has something planned 
> > around the same use case as well. 
>
> Hi Quirin, 
>
> Do you have anything planned on this? If not, I can have a look. 
> Please do let me know. 
>
>
Ping.

-Vijai Kumar K
 

> Thanks, 
> Vijai Kumar K 
>
> > 
> > Jan 
> > 
> > -- 
> > Siemens AG, Corporate Technology, CT RDA IOT SES-DE 
> > Corporate Competence Center Embedded Linux 
>

[-- Attachment #1.2: Type: text/html, Size: 4333 bytes --]