isar-exclude-docs vs. openjdk et al.

isar-exclude-docs vs. openjdk et al.

[Jan Kiszka]

Hi all,

there seems to be a conceptual issue with isar-exclude-docs purging
documents in its postinst hook: If other packages are configured after
isar-exclude-docs and those packages expects certain doc paths to be
still there, see openjdk [1], they will fail.

How to solve that?
 - maintain a list of conflicting packages in that recipe?
 - convert the package to a post-process hook?
 - find a way to ensure a compatible ordering when running our postinst?

Jan

[1] https://groups.google.com/g/isar-users/c/uIHgzvCGLwU/m/tuOchY6BAgAJ

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: isar-exclude-docs vs. openjdk et al.

[Henning Schild]

Hi,

not sure that doc purging is really allowed or whether we are messing
with stuff we better should not mess with.

Am Thu, 8 Apr 2021 20:54:28 +0200
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> Hi all,
> 
> there seems to be a conceptual issue with isar-exclude-docs purging
> documents in its postinst hook: If other packages are configured after
> isar-exclude-docs and those packages expects certain doc paths to be
> still there, see openjdk [1], they will fail.
> 
> How to solve that?
>  - maintain a list of conflicting packages in that recipe?

This is going to be cumbersome but when having a packet that needs its
docs, we probably should not install that cleaner.

>  - convert the package to a post-process hook?

would still break apt-get update of packages that need their docs

>  - find a way to ensure a compatible ordering when running our
> postinst?

again going to break apt-get

Looking at this one example it does not really need its docs, it just
expects them in "its own hack". My guess is that the problem can be
reproduced in a ubuntu container (where the no-docs stuff is coming
from), and can be used to report an issue so that the
update-alternatives script looks for the files conditionally. Or maybe
write a patch for that package and MR it on salsa directly.

If you are installing java you probably can store a few MB of docs ;)

Henning

> 
> Jan
> 
> [1]
> https://groups.google.com/g/isar-users/c/uIHgzvCGLwU/m/tuOchY6BAgAJ
> 

Re: isar-exclude-docs vs. openjdk et al.

[Jan Kiszka]

On 09.04.21 10:30, Henning Schild wrote:
> Hi,
> 
> not sure that doc purging is really allowed or whether we are messing
> with stuff we better should not mess with.
> 
> Am Thu, 8 Apr 2021 20:54:28 +0200
> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> 
>> Hi all,
>>
>> there seems to be a conceptual issue with isar-exclude-docs purging
>> documents in its postinst hook: If other packages are configured after
>> isar-exclude-docs and those packages expects certain doc paths to be
>> still there, see openjdk [1], they will fail.
>>
>> How to solve that?
>>  - maintain a list of conflicting packages in that recipe?
> 
> This is going to be cumbersome but when having a packet that needs its
> docs, we probably should not install that cleaner.
> 
>>  - convert the package to a post-process hook?
> 
> would still break apt-get update of packages that need their docs
> 
>>  - find a way to ensure a compatible ordering when running our
>> postinst?
> 
> again going to break apt-get
> 
> Looking at this one example it does not really need its docs, it just
> expects them in "its own hack". My guess is that the problem can be
> reproduced in a ubuntu container (where the no-docs stuff is coming
> from), and can be used to report an issue so that the
> update-alternatives script looks for the files conditionally. Or maybe
> write a patch for that package and MR it on salsa directly.
> 
> If you are installing java you probably can store a few MB of docs ;)
> 

Yeah, the case that triggered that is a bug, not a real scenario. I'm
lacking a feeling how common that combination is and if there might be
realistic combinations as well.

Jan

> Henning
> 
>>
>> Jan
>>
>> [1]
>> https://groups.google.com/g/isar-users/c/uIHgzvCGLwU/m/tuOchY6BAgAJ
>>
> 


-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: isar-exclude-docs vs. openjdk et al.

[Henning Schild]

Am Fri, 9 Apr 2021 15:01:48 +0200
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 09.04.21 10:30, Henning Schild wrote:
> > Hi,
> > 
> > not sure that doc purging is really allowed or whether we are
> > messing with stuff we better should not mess with.
> > 
> > Am Thu, 8 Apr 2021 20:54:28 +0200
> > schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >   
> >> Hi all,
> >>
> >> there seems to be a conceptual issue with isar-exclude-docs purging
> >> documents in its postinst hook: If other packages are configured
> >> after isar-exclude-docs and those packages expects certain doc
> >> paths to be still there, see openjdk [1], they will fail.
> >>
> >> How to solve that?
> >>  - maintain a list of conflicting packages in that recipe?  
> > 
> > This is going to be cumbersome but when having a packet that needs
> > its docs, we probably should not install that cleaner.
> >   
> >>  - convert the package to a post-process hook?  
> > 
> > would still break apt-get update of packages that need their docs
> >   
> >>  - find a way to ensure a compatible ordering when running our
> >> postinst?  
> > 
> > again going to break apt-get
> > 
> > Looking at this one example it does not really need its docs, it
> > just expects them in "its own hack". My guess is that the problem
> > can be reproduced in a ubuntu container (where the no-docs stuff is
> > coming from), and can be used to report an issue so that the
> > update-alternatives script looks for the files conditionally. Or
> > maybe write a patch for that package and MR it on salsa directly.
> > 
> > If you are installing java you probably can store a few MB of docs
> > ;) 
> 
> Yeah, the case that triggered that is a bug, not a real scenario. I'm
> lacking a feeling how common that combination is and if there might be
> realistic combinations as well.

My guess is that it would be best to try and mainline that package
right into debian. They could use it for their containers, offer it for
their users and will maintain the list of conflicts or detect such bugs.

As far as i remember the configs came from ubuntu, not sure they have
it packaged or just smuggle it into their containers.

That said, i bet there are packages that have runtime deps on (their)
docs. Doc readers or applications that opt for cat ing a file on
--help, probably not made up ...

Henning

> Jan
> 
> > Henning
> >   
> >>
> >> Jan
> >>
> >> [1]
> >> https://groups.google.com/g/isar-users/c/uIHgzvCGLwU/m/tuOchY6BAgAJ
> >>  
> >   
> 
> 

Re: isar-exclude-docs vs. openjdk et al.

[Silvano Cirujano Cuesta]

On 10/04/2021 10:46, [ext] Henning Schild wrote:
> Am Fri, 9 Apr 2021 15:01:48 +0200
> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> 
>> On 09.04.21 10:30, Henning Schild wrote:
>>> Hi,
>>>
>>> not sure that doc purging is really allowed or whether we are
>>> messing with stuff we better should not mess with.
>>>
>>> Am Thu, 8 Apr 2021 20:54:28 +0200
>>> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
>>>   
>>>> Hi all,
>>>>
>>>> there seems to be a conceptual issue with isar-exclude-docs purging
>>>> documents in its postinst hook: If other packages are configured
>>>> after isar-exclude-docs and those packages expects certain doc
>>>> paths to be still there, see openjdk [1], they will fail.
>>>>
>>>> How to solve that?
>>>>  - maintain a list of conflicting packages in that recipe?  
>>>
>>> This is going to be cumbersome but when having a packet that needs
>>> its docs, we probably should not install that cleaner.
>>>   
>>>>  - convert the package to a post-process hook?  
>>>
>>> would still break apt-get update of packages that need their docs
>>>   
>>>>  - find a way to ensure a compatible ordering when running our
>>>> postinst?  
>>>
>>> again going to break apt-get
>>>
>>> Looking at this one example it does not really need its docs, it
>>> just expects them in "its own hack". My guess is that the problem
>>> can be reproduced in a ubuntu container (where the no-docs stuff is
>>> coming from), and can be used to report an issue so that the
>>> update-alternatives script looks for the files conditionally. Or
>>> maybe write a patch for that package and MR it on salsa directly.
>>>
>>> If you are installing java you probably can store a few MB of docs
>>> ;) 
>>
>> Yeah, the case that triggered that is a bug, not a real scenario. I'm
>> lacking a feeling how common that combination is and if there might be
>> realistic combinations as well.
> 
> My guess is that it would be best to try and mainline that package
> right into debian. They could use it for their containers, offer it for
> their users and will maintain the list of conflicts or detect such bugs.

It's not only an ISAR issue.
Just try to install openjdk-11-jre-headless on a debian:buster-slim image to reproduce it.

I'll report the bug on the Debian openjdk package, since the images being provided by Docker can be considered the de-facto standard (even provided by Debian maintainers Tianon Gravi [1] and Paul Tagliamonte [2]).

I'd rather wait for them to react to it before deciding which way we go.

Curious about how those images are being created? See here: https://github.com/debuerreotype/debuerreotype

  Silvano

[1] https://qa.debian.org/developer.php?login=tianon
[2] https://qa.debian.org/developer.php?login=paultag

> 
> As far as i remember the configs came from ubuntu, not sure they have
> it packaged or just smuggle it into their containers.
> 
> That said, i bet there are packages that have runtime deps on (their)
> docs. Doc readers or applications that opt for cat ing a file on
> --help, probably not made up ...
> 
> Henning
> 
>> Jan
>>
>>> Henning
>>>   
>>>>
>>>> Jan
>>>>
>>>> [1]
>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fisar-users%2Fc%2FuIHgzvCGLwU%2Fm%2FtuOchY6BAgAJ&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C86e2085237e3487d4c8608d8fbfea912%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637536418464481337%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=f%2F4dVdp5tYLFm5KbqvFdNE7rXLMUiMP5hnI8oGcmPC4%3D&amp;reserved=0
>>>>  
>>>   
>>
>>
> 

Re: isar-exclude-docs vs. openjdk et al.

[Silvano Cirujano Cuesta]

On 12/04/2021 10:47, [ext] Silvano Cirujano Cuesta wrote:
> 
> 
> On 10/04/2021 10:46, [ext] Henning Schild wrote:
>> Am Fri, 9 Apr 2021 15:01:48 +0200
>> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
>>
>>> On 09.04.21 10:30, Henning Schild wrote:
>>>> Hi,
>>>>
>>>> not sure that doc purging is really allowed or whether we are
>>>> messing with stuff we better should not mess with.
>>>>
>>>> Am Thu, 8 Apr 2021 20:54:28 +0200
>>>> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
>>>>   
>>>>> Hi all,
>>>>>
>>>>> there seems to be a conceptual issue with isar-exclude-docs purging
>>>>> documents in its postinst hook: If other packages are configured
>>>>> after isar-exclude-docs and those packages expects certain doc
>>>>> paths to be still there, see openjdk [1], they will fail.
>>>>>
>>>>> How to solve that?
>>>>>  - maintain a list of conflicting packages in that recipe?  
>>>>
>>>> This is going to be cumbersome but when having a packet that needs
>>>> its docs, we probably should not install that cleaner.
>>>>   
>>>>>  - convert the package to a post-process hook?  
>>>>
>>>> would still break apt-get update of packages that need their docs
>>>>   
>>>>>  - find a way to ensure a compatible ordering when running our
>>>>> postinst?  
>>>>
>>>> again going to break apt-get
>>>>
>>>> Looking at this one example it does not really need its docs, it
>>>> just expects them in "its own hack". My guess is that the problem
>>>> can be reproduced in a ubuntu container (where the no-docs stuff is
>>>> coming from), and can be used to report an issue so that the
>>>> update-alternatives script looks for the files conditionally. Or
>>>> maybe write a patch for that package and MR it on salsa directly.
>>>>
>>>> If you are installing java you probably can store a few MB of docs
>>>> ;) 
>>>
>>> Yeah, the case that triggered that is a bug, not a real scenario. I'm
>>> lacking a feeling how common that combination is and if there might be
>>> realistic combinations as well.
>>
>> My guess is that it would be best to try and mainline that package
>> right into debian. They could use it for their containers, offer it for
>> their users and will maintain the list of conflicts or detect such bugs.
> 
> It's not only an ISAR issue.
> Just try to install openjdk-11-jre-headless on a debian:buster-slim image to reproduce it.
> 
> I'll report the bug on the Debian openjdk package, since the images being provided by Docker can be considered the de-facto standard (even provided by Debian maintainers Tianon Gravi [1] and Paul Tagliamonte [2]).

No need to do it: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955619 :-)

I wonder if the initial issue was the installation of openjdk on a container image generated with ISAR or on the host installation.
I suspect that it's in the host and we've faced it only due to the regression I introduced :-(

If we really need the installation on a container image, then there's a fix proposal in the Debian bug that we might need to apply on openjdk.

  Silvano

> 
> I'd rather wait for them to react to it before deciding which way we go.
> 
> Curious about how those images are being created? See here: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdebuerreotype%2Fdebuerreotype&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C3abb71e1f482453b80e008d8fd8f9a9b%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637538140518927690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=yG9G6p%2FL3KavzwYQB0lsyRp5VSG0ylnZmZFy7ppo54g%3D&amp;reserved=0
> 
>   Silvano
> 
> [1] https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqa.debian.org%2Fdeveloper.php%3Flogin%3Dtianon&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C3abb71e1f482453b80e008d8fd8f9a9b%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637538140518927690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WBxXswRJaurv%2BQUukatMQw61kP0m6c6%2BsoitiCh5LQs%3D&amp;reserved=0
> [2] https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqa.debian.org%2Fdeveloper.php%3Flogin%3Dpaultag&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C3abb71e1f482453b80e008d8fd8f9a9b%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637538140518927690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=s4HIASCcpLTXNuZnDDOI%2BqrIhzrGLx6RAcd8WdK3u7U%3D&amp;reserved=0
> 
>>
>> As far as i remember the configs came from ubuntu, not sure they have
>> it packaged or just smuggle it into their containers.
>>
>> That said, i bet there are packages that have runtime deps on (their)
>> docs. Doc readers or applications that opt for cat ing a file on
>> --help, probably not made up ...
>>
>> Henning
>>
>>> Jan
>>>
>>>> Henning
>>>>   
>>>>>
>>>>> Jan
>>>>>
>>>>> [1]
>>>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fisar-users%2Fc%2FuIHgzvCGLwU%2Fm%2FtuOchY6BAgAJ&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C3abb71e1f482453b80e008d8fd8f9a9b%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637538140518927690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=kbA93XVRKColo6QKgCgdxNNgzmZMLlTIjgIJ1oPkEAU%3D&amp;reserved=0
>>>>>  
>>>>   
>>>
>>>
>>
> 

Re: isar-exclude-docs vs. openjdk et al.

[Henning Schild]

Am Mon, 12 Apr 2021 10:47:14 +0200
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cirujano-cuesta@siemens.com>:

> On 10/04/2021 10:46, [ext] Henning Schild wrote:
> > Am Fri, 9 Apr 2021 15:01:48 +0200
> > schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >   
> >> On 09.04.21 10:30, Henning Schild wrote:  
> >>> Hi,
> >>>
> >>> not sure that doc purging is really allowed or whether we are
> >>> messing with stuff we better should not mess with.
> >>>
> >>> Am Thu, 8 Apr 2021 20:54:28 +0200
> >>> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >>>     
> >>>> Hi all,
> >>>>
> >>>> there seems to be a conceptual issue with isar-exclude-docs
> >>>> purging documents in its postinst hook: If other packages are
> >>>> configured after isar-exclude-docs and those packages expects
> >>>> certain doc paths to be still there, see openjdk [1], they will
> >>>> fail.
> >>>>
> >>>> How to solve that?
> >>>>  - maintain a list of conflicting packages in that recipe?    
> >>>
> >>> This is going to be cumbersome but when having a packet that needs
> >>> its docs, we probably should not install that cleaner.
> >>>     
> >>>>  - convert the package to a post-process hook?    
> >>>
> >>> would still break apt-get update of packages that need their docs
> >>>     
> >>>>  - find a way to ensure a compatible ordering when running our
> >>>> postinst?    
> >>>
> >>> again going to break apt-get
> >>>
> >>> Looking at this one example it does not really need its docs, it
> >>> just expects them in "its own hack". My guess is that the problem
> >>> can be reproduced in a ubuntu container (where the no-docs stuff
> >>> is coming from), and can be used to report an issue so that the
> >>> update-alternatives script looks for the files conditionally. Or
> >>> maybe write a patch for that package and MR it on salsa directly.
> >>>
> >>> If you are installing java you probably can store a few MB of docs
> >>> ;)   
> >>
> >> Yeah, the case that triggered that is a bug, not a real scenario.
> >> I'm lacking a feeling how common that combination is and if there
> >> might be realistic combinations as well.  
> > 
> > My guess is that it would be best to try and mainline that package
> > right into debian. They could use it for their containers, offer it
> > for their users and will maintain the list of conflicts or detect
> > such bugs.  
> 
> It's not only an ISAR issue.
> Just try to install openjdk-11-jre-headless on a debian:buster-slim
> image to reproduce it.

Yes that particular slim container contains a similar hack to remove
docs. But that is not coming from any package ... so Isar is even
better here ;). Maybe upstream should have a "slim" package and they
could maintain conflicts.

root@1b57a090701a:/etc/dpkg/dpkg.cfg.d# dpkg -S
/etc/dpkg/dpkg.cfg.d/docker dpkg-query: no path found matching pattern
/etc/dpkg/dpkg.cfg.d/docker

Henning

> I'll report the bug on the Debian openjdk package, since the images
> being provided by Docker can be considered the de-facto standard
> (even provided by Debian maintainers Tianon Gravi [1] and Paul
> Tagliamonte [2]).
> 
> I'd rather wait for them to react to it before deciding which way we
> go.
> 
> Curious about how those images are being created? See here:
> https://github.com/debuerreotype/debuerreotype
> 
>   Silvano
> 
> [1] https://qa.debian.org/developer.php?login=tianon
> [2] https://qa.debian.org/developer.php?login=paultag
> 
> > 
> > As far as i remember the configs came from ubuntu, not sure they
> > have it packaged or just smuggle it into their containers.
> > 
> > That said, i bet there are packages that have runtime deps on
> > (their) docs. Doc readers or applications that opt for cat ing a
> > file on --help, probably not made up ...
> > 
> > Henning
> >   
> >> Jan
> >>  
> >>> Henning
> >>>     
> >>>>
> >>>> Jan
> >>>>
> >>>> [1]
> >>>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fisar-users%2Fc%2FuIHgzvCGLwU%2Fm%2FtuOchY6BAgAJ&amp;data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C86e2085237e3487d4c8608d8fbfea912%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637536418464481337%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=f%2F4dVdp5tYLFm5KbqvFdNE7rXLMUiMP5hnI8oGcmPC4%3D&amp;reserved=0
> >>>>    
> >>>     
> >>
> >>  
> >   
>