RFC: base-apt caching improvements

[Henning Schild <henning.schild@siemens.com>]

Hi,

i did not really like the current approach to how we cache, because i
thought we can and should do that somehow transparent and route all
"apt" downloads through a proxy that will reprepro all files as the are
requested.
I did not find a proxy that can do that, maybe i did not look hard
enough.

So i came back to the currently implemented model. Collect the files
from the rootfs-es after we are all done. The way that is currently
done is getting them from the apt cache. Unfortunately that is not
guaranteed to work. Any package we install could mess with apt-config
and therefore the caching of the rootfs-es. (i.e. something
like /etc/apt/apt.conf.d/docker-clean you will find a container images)

So that cache can not be trusted. And that cache does not work for
source packages and can not fulfill our needs with regards to full
caching/archiving.

So instead we should go and download all debs and sources of the
currently installed packages of all rootfs-es (target and its
buildchroots) explicitly and reprepro from there.

Here is what this could look like:

rm -rf /tmp/foo
cd /tmp/foo
dpkg -l | grep "^ii" | awk '{print $2"="$3}' | xargs apt-get -y download
### reprepro *.deb
rm -rf /tmp/foo-src
cd /tmp/foo-src
dpkg -l | grep "^ii" | awk '{print $2"="$3}' | xargs apt-get -y source --download-only
### reprepro *.dsc

This still lacks filtering out all the packages from isar-apt but could
be an improvement to our current way. We do not need to trust those fragile
caches because we are explicit. We can do sources ... and should probably
just do all of them anyways.
For selective sources we could "apt-get source" into a central dldir and
copy the sources to the recipe workdir. Would be a matter of mounting that
shared dir and changing do_apt_fetch to use that staging dir to leave a copy.
In fact we (Siemens) want all sources, and it might be a sane default for
others as well.

Henning