Lightning talk about ISAR

Lightning talk about ISAR

[Moessbauer, Felix]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]

Dear subscribers,

we plan to give a lightning talk about recent changes in ISAR at the "Linux Community Event" conference (Siemens internal).
The talk should give an overview about major changes and features, added to ISAR since the last year.

Below you will find a list of topics we plan to present.
Are there any further topics which we should cover?

- distro support
  - added debian bullseye
  - added ubuntu (focal)
- target support
  - added sample RISC-V target
- image formats
  - added OVA (VMWare) image type
  - added CPIO image type (initramfs like filesystem)
  - generate OCI (container) filesystem
- Performance optimizations
  - SState
  - CCache (Upcoming)

Minor changes:
- Install prebuild DPKD packages from third party sources
- Support for apt sources in "Absolute Component Format"
- Package building
  - DEBIAN_BUILD_DEPENDS as BB parameter
  - DEBIAN_CONFLICTS as BB parameter
- git as default patchtool
- Warn if FS has Y2038 issue
- Changes to ISAR_RELEASE_CMD (upcoming)

Best regards,
Felix

[-- Attachment #2: Type: text/html, Size: 3630 bytes --]

Re: Lightning talk about ISAR

[Jan Kiszka]

On 25.11.21 11:42, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
> Dear subscribers,
> 

Dear sender: please no HTML on public mailing lists.

> 
> we plan to give a lightning talk about recent changes in ISAR at the
> “Linux Community Event” conference (Siemens internal).
> The talk should give an overview about major changes and features, added
> to ISAR since the last year.
> 
> Below you will find a list of topics we plan to present.
> Are there any further topics which we should cover?

Which timeframe would you cover? Roughly last 2 years?

> 
> - distro support
>   - added debian bullseye
>   - added ubuntu (focal)
> - target support
>   - added sample RISC-V target
> - image formats
>   - added OVA (VMWare) image type
>   - added CPIO image type (initramfs like filesystem)
>   - generate OCI (container) filesystem

+ containerized SDK

> - Performance optimizations
>   - SState
>   - CCache (Upcoming)

sstate is upcoming, ccache is in.

> 
> Minor changes:
> - Install prebuild DPKD packages from third party sources
> - Support for apt sources in "Absolute Component Format"
> - Package building
>   - DEBIAN_BUILD_DEPENDS as BB parameter
>   - DEBIAN_CONFLICTS as BB parameter
> - git as default patchtool
> - Warn if FS has Y2038 issue
> - Changes to ISAR_RELEASE_CMD (upcoming)

- deb-src caching
- deb_debianize received quite a few detail improvements
- package build for 32-bit compat archs
- TF-A and OP-TEE packaging support
- new testsuite (avocado based) -> relevant for contributors

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: Lightning talk about ISAR

[Henning Schild]

Am Thu, 25 Nov 2021 11:42:00 +0100
schrieb "Moessbauer, Felix (T RDA IOT SES-DE)"
<felix.moessbauer@siemens.com>:

> Dear subscribers,
> 
>  
> 
> we plan to give a lightning talk about recent changes in ISAR at the
> “Linux Community Event” conference (Siemens internal).

lightning talk means we got a 15min slot, it seems too much content to
me, or maybe we get a longer slot for that talk

> The talk should give an overview about major changes and features,
> added to ISAR since the last year.

Besides features we see more and more adoption and users being pretty
happy to build with ISAR.
That includes Siemens internal communication and what we saw coming
from Intel and others. Users who are happy and do not even show up on
this list. Might be worth putting on a slide.

>  
> 
> Below you will find a list of topics we plan to present.
> 
> Are there any further topics which we should cover?
> 
>  
> 
> - distro support
> 
>   - added debian bullseye
> 
>   - added ubuntu (focal)
> 
> - target support
> 
>   - added sample RISC-V target
> 
> - image formats
> 
>   - added OVA (VMWare) image type
> 
>   - added CPIO image type (initramfs like filesystem)
> 
>   - generate OCI (container) filesystem
> 
> - Performance optimizations
> 
>   - SState
> 
>   - CCache (Upcoming)


We have the sbuild/schroot story (upcoming). I did not yet try it but
guess it will be a performance topic (because of less serialization)
and a quality topic because of "Build-Depends".

Seeing the so far very positive adoption i would conclude that
"performance" it the #1 in all that has happened. With all the other
topics being "we care to improve and cater".

regards,
Henning

>  
> 
> Minor changes:
> 
> - Install prebuild DPKD packages from third party sources
> 
> - Support for apt sources in "Absolute Component Format"
> 
> - Package building
> 
>   - DEBIAN_BUILD_DEPENDS as BB parameter
> 
>   - DEBIAN_CONFLICTS as BB parameter
> 
> - git as default patchtool
> 
> - Warn if FS has Y2038 issue
> 
> - Changes to ISAR_RELEASE_CMD (upcoming)
> 
> Best regards,
> Felix
> 

Re: Lightning talk about ISAR

[Henning Schild]

We also have a bunch of commits for reproducible builds, i am afraid
still (upcoming).

I carry them as "henning/staging10"
https://github.com/henning-schild-work/isar/tree/henning/staging10
Yes ... two digit open "MRs" ...

Might be worth mentioning because "reproducible builds" will be a
keynote talk on that event.

OT

i stopped all rebasing and pinging because we need that sstate to speed
up and get all those open MRs merged. Some of them introduce tests and
i decided to hold them back after avocado. Will be happy to re-send
once sstate is in and we gain some much needed speed.

Henning

Am Thu, 25 Nov 2021 21:17:05 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> Am Thu, 25 Nov 2021 11:42:00 +0100
> schrieb "Moessbauer, Felix (T RDA IOT SES-DE)"
> <felix.moessbauer@siemens.com>:
> 
> > Dear subscribers,
> > 
> >  
> > 
> > we plan to give a lightning talk about recent changes in ISAR at the
> > “Linux Community Event” conference (Siemens internal).  
> 
> lightning talk means we got a 15min slot, it seems too much content to
> me, or maybe we get a longer slot for that talk
> 
> > The talk should give an overview about major changes and features,
> > added to ISAR since the last year.  
> 
> Besides features we see more and more adoption and users being pretty
> happy to build with ISAR.
> That includes Siemens internal communication and what we saw coming
> from Intel and others. Users who are happy and do not even show up on
> this list. Might be worth putting on a slide.
> 
> >  
> > 
> > Below you will find a list of topics we plan to present.
> > 
> > Are there any further topics which we should cover?
> > 
> >  
> > 
> > - distro support
> > 
> >   - added debian bullseye
> > 
> >   - added ubuntu (focal)
> > 
> > - target support
> > 
> >   - added sample RISC-V target
> > 
> > - image formats
> > 
> >   - added OVA (VMWare) image type
> > 
> >   - added CPIO image type (initramfs like filesystem)
> > 
> >   - generate OCI (container) filesystem
> > 
> > - Performance optimizations
> > 
> >   - SState
> > 
> >   - CCache (Upcoming)  
> 
> 
> We have the sbuild/schroot story (upcoming). I did not yet try it but
> guess it will be a performance topic (because of less serialization)
> and a quality topic because of "Build-Depends".
> 
> Seeing the so far very positive adoption i would conclude that
> "performance" it the #1 in all that has happened. With all the other
> topics being "we care to improve and cater".
> 
> regards,
> Henning
> 
> >  
> > 
> > Minor changes:
> > 
> > - Install prebuild DPKD packages from third party sources
> > 
> > - Support for apt sources in "Absolute Component Format"
> > 
> > - Package building
> > 
> >   - DEBIAN_BUILD_DEPENDS as BB parameter
> > 
> >   - DEBIAN_CONFLICTS as BB parameter
> > 
> > - git as default patchtool
> > 
> > - Warn if FS has Y2038 issue
> > 
> > - Changes to ISAR_RELEASE_CMD (upcoming)
> > 
> > Best regards,
> > Felix
> >   
> 

Re: Lightning talk about ISAR

[Henning Schild]

Hi all,

the talk went well and i think we can conclude that Isar adoption is
right on track! We have many stakeholders building their firmwares with
Isar and being pretty happy with it!

The talk did focus on performance improvements like sstate and ccache
and did also mention some other changes.

We did have a poll at the end to see where users see room for
improvement. I hope Felix will share the results ... but the number one
topic was "documentation" if i recall correctly.

Short conclusion is ... "Isar is great and works", performance was a
big deal but we are on the right track ... hopefully. But new users and
early adopters rightfully dislike the docs. Surpringly the API
changelog seems fine ... even if we have to break things every once i a
while!

I would take this as a very good indication, but we should not stop
improving. While Isar works well all devs know that its core has many
problems that deserve proper fixes!

regards,
Henning

Am Thu, 25 Nov 2021 11:42:00 +0100
schrieb "Moessbauer, Felix (T RDA IOT SES-DE)"
<felix.moessbauer@siemens.com>:

> Dear subscribers,
> 
>  
> 
> we plan to give a lightning talk about recent changes in ISAR at the
> “Linux Community Event” conference (Siemens internal).
> 
> The talk should give an overview about major changes and features,
> added to ISAR since the last year.
> 
>  
> 
> Below you will find a list of topics we plan to present.
> 
> Are there any further topics which we should cover?
> 
>  
> 
> - distro support
> 
>   - added debian bullseye
> 
>   - added ubuntu (focal)
> 
> - target support
> 
>   - added sample RISC-V target
> 
> - image formats
> 
>   - added OVA (VMWare) image type
> 
>   - added CPIO image type (initramfs like filesystem)
> 
>   - generate OCI (container) filesystem
> 
> - Performance optimizations
> 
>   - SState
> 
>   - CCache (Upcoming)
> 
>  
> 
> Minor changes:
> 
> - Install prebuild DPKD packages from third party sources
> 
> - Support for apt sources in "Absolute Component Format"
> 
> - Package building
> 
>   - DEBIAN_BUILD_DEPENDS as BB parameter
> 
>   - DEBIAN_CONFLICTS as BB parameter
> 
> - git as default patchtool
> 
> - Warn if FS has Y2038 issue
> 
> - Changes to ISAR_RELEASE_CMD (upcoming)
> 
> Best regards,
> Felix
> 

Re: Lightning talk about ISAR

[Baurzhan Ismagulov]

On Thu, Dec 09, 2021 at 11:02:01PM +0100, Henning Schild wrote:
> the talk went well and i think we can conclude that Isar adoption is
> right on track! We have many stakeholders building their firmwares with
> Isar and being pretty happy with it!

Glad to hear that!


> We did have a poll at the end to see where users see room for
> improvement. I hope Felix will share the results ... but the number one
> topic was "documentation" if i recall correctly.

Yes, I agree it's time to prioritize this continuous TODO item. If you have a
list of specific topics other than known ones, it would help.


> Surpringly the API
> changelog seems fine ... even if we have to break things every once i a
> while!

This is also good to know.


On Fri, Nov 26, 2021 at 09:20:48PM +0100, Henning Schild wrote:
> i stopped all rebasing and pinging because we need that sstate to speed
> up and get all those open MRs merged. Some of them introduce tests and
> i decided to hold them back after avocado. Will be happy to re-send
> once sstate is in and we gain some much needed speed.

Now that avocado and sstate cache is in, if any other patches have priority,
please give us a hint.


With kind regards,
Baurzhan.

RE: Lightning talk about ISAR

[Moessbauer, Felix]

> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Baurzhan Ismagulov
> Sent: Friday, December 10, 2021 11:57 AM
> To: isar-users@googlegroups.com
> Subject: Re: Lightning talk about ISAR
> 
> On Thu, Dec 09, 2021 at 11:02:01PM +0100, Henning Schild wrote:
> > the talk went well and i think we can conclude that Isar adoption is
> > right on track! We have many stakeholders building their firmwares
> > with Isar and being pretty happy with it!
> 
> Glad to hear that!
> 
> 
> > We did have a poll at the end to see where users see room for
> > improvement. I hope Felix will share the results ... but the number
> > one topic was "documentation" if i recall correctly.
> 
> Yes, I agree it's time to prioritize this continuous TODO item. If you have a list of
> specific topics other than known ones, it would help.

Sure: the (unfiltered) answers to the question "What feature du you miss in ISAR?" are: (~ordered by count)
- documentation
- quickstart guide / tutorial
- swu image generation
- minimal initramfs img
- list of available packets
- uefi secure boot
- less steep learning curve
- tmp2 support
- oss archive generation

Some of them are already implemented, but that nails it down to the documentation aspect.
I also encouraged the listeners to mention the topics on the ML 😉

Felix

> 
> 
> > Surpringly the API
> > changelog seems fine ... even if we have to break things every once i
> > a while!
> 
> This is also good to know.
> 
> 
> On Fri, Nov 26, 2021 at 09:20:48PM +0100, Henning Schild wrote:
> > i stopped all rebasing and pinging because we need that sstate to
> > speed up and get all those open MRs merged. Some of them introduce
> > tests and i decided to hold them back after avocado. Will be happy to
> > re-send once sstate is in and we gain some much needed speed.
> 
> Now that avocado and sstate cache is in, if any other patches have priority,
> please give us a hint.
> 
> 
> With kind regards,
> Baurzhan.
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-
> users%2F20211210105632.GK22100%2540yssyq.m.ilbers.de&amp;data=04%7C0
> 1%7Cfelix.moessbauer%40siemens.com%7Ce329d203a2e0433ea78408d9bbcbb
> cdc%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C63774730688256
> 6758%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=zXUlJ3f9171D9STCf7
> hCvzKpV%2F1GCi4LoTNf6KJcsQE%3D&amp;reserved=0.

Re: Lightning talk about ISAR

[Jan Kiszka]

On 10.12.21 11:56, Baurzhan Ismagulov wrote:
> On Thu, Dec 09, 2021 at 11:02:01PM +0100, Henning Schild wrote:
>> the talk went well and i think we can conclude that Isar adoption is
>> right on track! We have many stakeholders building their firmwares with
>> Isar and being pretty happy with it!
> 
> Glad to hear that!
> 
> 
>> We did have a poll at the end to see where users see room for
>> improvement. I hope Felix will share the results ... but the number one
>> topic was "documentation" if i recall correctly.
> 
> Yes, I agree it's time to prioritize this continuous TODO item. If you have a
> list of specific topics other than known ones, it would help.
> 
> 
>> Surpringly the API
>> changelog seems fine ... even if we have to break things every once i a
>> while!
> 
> This is also good to know.
> 
> 
> On Fri, Nov 26, 2021 at 09:20:48PM +0100, Henning Schild wrote:
>> i stopped all rebasing and pinging because we need that sstate to speed
>> up and get all those open MRs merged. Some of them introduce tests and
>> i decided to hold them back after avocado. Will be happy to re-send
>> once sstate is in and we gain some much needed speed.
> 
> Now that avocado and sstate cache is in, if any other patches have priority,
> please give us a hint.
> 

IMHO: Stabilization patches so that 0.8-rc1 can be tagged and then 0.8.
After that, we should look into sbuild and other refactorings that are
pending.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: Lightning talk about ISAR

[Baurzhan Ismagulov]

On Fri, Dec 10, 2021 at 12:23:32PM +0100, Jan Kiszka wrote:
> > Now that avocado and sstate cache is in, if any other patches have priority,
> > please give us a hint.
> 
> IMHO: Stabilization patches so that 0.8-rc1 can be tagged and then 0.8.
> After that, we should look into sbuild and other refactorings that are
> pending.

Right, good point. We've already started checking what needs to be done and
will post our suggestions.

With kind regards,
Baurzhan.

Re: Lightning talk about ISAR

[Henning Schild]

Am Fri, 10 Dec 2021 12:36:10 +0100
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Fri, Dec 10, 2021 at 12:23:32PM +0100, Jan Kiszka wrote:
> > > Now that avocado and sstate cache is in, if any other patches
> > > have priority, please give us a hint.  
> > 
> > IMHO: Stabilization patches so that 0.8-rc1 can be tagged and then
> > 0.8. After that, we should look into sbuild and other refactorings
> > that are pending.  
> 
> Right, good point. We've already started checking what needs to be
> done and will post our suggestions.

I hope there will be some sort of -rc phase for another release.
Because i do have double digits MRs open at the moment.

I would be happy to rebase and stuff, i did stop pushing because i
wanted sstate in first.

There are things around VMs, containers, and so on, all kind of held
back for sstate and avocado, but still pretty valid and already
reviewed.

Henning

> With kind regards,
> Baurzhan.
> 

Re: Lightning talk about ISAR

[Baurzhan Ismagulov]

On Mon, Dec 13, 2021 at 10:17:15PM +0100, Henning Schild wrote:
> I hope there will be some sort of -rc phase for another release.
> Because i do have double digits MRs open at the moment.
> 
> I would be happy to rebase and stuff, i did stop pushing because i
> wanted sstate in first.
> 
> There are things around VMs, containers, and so on, all kind of held
> back for sstate and avocado, but still pretty valid and already
> reviewed.

Thanks all for your input. Yes, there will be a stabilization phase and rc(s).

With kind regards,
Baurzhan.

RE: Lightning talk about ISAR

[Schmidt, Adriaan]

Baurzhan Ismagulov, 10. Dezember 2021 12:36:
> On Fri, Dec 10, 2021 at 12:23:32PM +0100, Jan Kiszka wrote:
> > > Now that avocado and sstate cache is in, if any other patches have
> priority,
> > > please give us a hint.
> >
> > IMHO: Stabilization patches so that 0.8-rc1 can be tagged and then 0.8.
> > After that, we should look into sbuild and other refactorings that are
> > pending.
> 
> Right, good point. We've already started checking what needs to be done and
> will post our suggestions.

I still have some follow-ups on the sstate-cache feature:
- Test case that was in the original patch series. This builds on a testsuite
  refactoring currently in discussion. Once that is settled I can rebase and
  resubmit.
- Documentation: I don't have it yet, but will propose something.

And on sstate-cache, but not quite ready:
- Maintenance script for analysis and cleanup of caches, which we're using
  in CI in our downstream project. IT supports persistent/shared caches
  on filesystems, HTTP (webdav) servers, and S3 buckets.
  Still work in progress, but could be a valuable utility for Isar.

Other work in progress to look forward to, which should then come after this release:
- Refactoring of the (un)mounting logic (was already on the list, mostly works,
  but finalizing/testing will be some more effort, and will probably have
  collisions with other work like sbuilder (?))
- Extending image types (IMAGE_FSTYPES) to support OE-style type dependencies
  and image conversions (not yet on the list). This could greatly improve the
  image building/combining experience, but will of course break some interfaces
  (esp. in downstream layers that have their own image types). Will try to
  post a preview soon.

Adriaan