Re: [PATCH 5/6] meta/dpkg: add dpkg-custom class

[Alexander Smirnov <asmirnov@ilbers.de>]

On 09/08/2017 11:42 AM, Henning Schild wrote:
> Am Fri, 8 Sep 2017 11:31:23 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> On 09/08/2017 11:15 AM, Henning Schild wrote:
>>> Am Mon, 4 Sep 2017 18:36:23 +0300
>>> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
>>>    
>>>> On 08/30/2017 10:03 PM, Henning Schild wrote:
>>>>> Issues:
>>>>> 1. full customizations of the images is hard to impossible to
>>>>> realize in a layer without touching Isar
>>>>> 1.1. there is no easy way to just copy a file into the image
>>>>> 1.2. configuration (passwords, groups, cfg-files changes) can not
>>>>> be done in a layer, there is no way too hook into multistrap or
>>>>> the configure-script
>>>>>
>>>>> Change:
>>>>> Introduce a class that lets users create custom debian packages on
>>>>> the fly, without having to create a /debian directory and actually
>>>>> building. That allows you to pull in debian-dependencies, you
>>>>> could have a package that has no content and is just there to
>>>>> install what you need for a feature of your product.
>>>>> Using package hooks (preinst, postinst ..) you can configure
>>>>> pretty much all you want when installing the package.
>>>>> The package can contain actual payload as well, basically any
>>>>> files that come from "somewhere else". Say binary data like
>>>>> wallpapers, sound files or application binaries.
>>>>>
>>>>> Impact:
>>>>> This patch addresses the metioned issue in a way that uses debian
>>>>> mechanism. All the customizations will enjoy features like
>>>>> - collission protection (multiple packages providing the same
>>>>> file)
>>>>> - config file protection
>>>>> - versioning and the ability to deploy your changes in an
>>>>> updateable way
>>>>>
>>>>> This patch introduces a major new feature to Isar.
>>>>>
>>>>> This class introduces a new class for building debian packages on
>>>>> the fly. They can basically contain anything from random sources,
>>>>> where building happens outside of Isar. It also allows to create
>>>>> meta-packages that contain nothing but pull in dependencies, once
>>>>> all our packets come in via multistrap that will come in handy.
>>>>> For rootfs configuration you would use post- and pre- scripts just
>>>>> like regular debian packages do.
>>>>>       
>>>>
>>>> Good idea.
>>>>   
>>>>> Signed-off-by: Henning Schild <henning.schild@siemens.com>
>>>>> ---
>>>>>     meta/classes/dpkg-custom.bbclass | 57
>>>>> ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57
>>>>> insertions(+) create mode 100644 meta/classes/dpkg-custom.bbclass
>>>>>
>>>>> diff --git a/meta/classes/dpkg-custom.bbclass
>>>>> b/meta/classes/dpkg-custom.bbclass new file mode 100644
>>>>> index 0000000..e4e743f
>>>>> --- /dev/null
>>>>> +++ b/meta/classes/dpkg-custom.bbclass
>>>>> @@ -0,0 +1,57 @@
>>>>> +# This software is a part of ISAR.
>>>>> +# Copyright (C) 2017 Siemens AG
>>>>> +
>>>>> +inherit dpkg
>>>>> +
>>>>> +DEBIAN_DEPENDS ?= ""
>>>>> +MAINTAINER ?= "FIXME Unknown maintainer"
>>>>> +
>>>>> +D = "${WORKDIR}/image/"
>>>>> +
>>>>> +# Populate folder that will be picked up as package
>>>>> +# TODO this should be called 'do_install'
>>>>> +do_populate_package() {
>>>>> +	bbnote "Put your files for this package in ${D}"
>>>>> +}
>>>>> +
>>>>> +addtask populate_package after do_unpack before
>>>>> do_deb_package_prepare +
>>>>> +# so we can put hooks etc. in there already
>>>>> +do_populate_package[dirs] = "${D}/DEBIAN"
>>>>> +
>>>>> +do_deb_package_prepare() {
>>>>> +	cat<<-__EOF__ > ${D}/DEBIAN/control
>>>>> +		Package: ${PN}
>>>>> +		Architecture: `dpkg --print-architecture`
>>>>> +		Section: misc
>>>>> +		Priority: optional
>>>>> +		Maintainer: ${MAINTAINER}
>>>>> +		Depends: `echo ${DEBIAN_DEPENDS} | tr '[:blank:]'
>>>>> ','`
>>>>> +		Version: ${PV}+isar
>>>>> +		Description: ${DESCRIPTION}
>>>>> +	__EOF__
>>>>> +	for t in pre post
>>>>> +	do
>>>>> +		for a in inst rm
>>>>> +		do
>>>>> +			chmod -f +x ${D}/DEBIAN/${t}${a} || true
>>>>> +		done
>>>>> +	done
>>>>> +}
>>>>> +
>>>>> +addtask deb_package_prepare after do_populate_package before
>>>>> do_deb_package_conffiles +
>>>>> +do_deb_package_conffiles() {
>>>>> +	CONFFILES=${D}/DEBIAN/conffiles
>>>>> +	find ${D} -type f -path '*/etc/*' | sed -e 's|^${D}|/|'
>>>>>>> $CONFFILES
>>>>> +	test -s $CONFFILES || rm $CONFFILES
>>>>> +}
>>>>> +
>>>>> +addtask deb_package_conffiles after do_deb_package_prepare before
>>>>> do_deb_package +
>>>>> +do_deb_package() {
>>>>> +	sudo chown -R root:root ${D}/DEBIAN/
>>>>> +	sudo dpkg-deb --build ${D} ${WORKDIR}
>>>>> +}
>>>>> +
>>>>> +addtask deb_package after do_deb_package_conffiles before
>>>>> do_install
>>>>
>>>> If I got this correctly, the difference from dpkg-debian flow is
>>>> that you generate 'debian' folder on the fly. So:
>>>
>>> Well it is a capital "DEBIAN" folder that contains the package
>>> metadata and no build instructions like 'debian'.
>>>    
>>>> 1. Why we need another do_deb_package, while do_build from dpkg
>>>> class does the same?
>>>
>>> do_build does not do the same. It eventually calls
>>> 'dpkg-buildpackage' which does "configure; make; make install; make
>>> deb"
>>>
>>> Using the same language this step is just "make deb" and it uses the
>>> debian tool 'dpkg-deb'.
>>>
>>> Custom steps for "configure; make;" could be added in the future.
>>> "make install" is do_populate_package.
>>
>> Ok, I see now. But why it's done by host dpkg-deb tool? This should
>> be done inside buildchroot to keep version compatibility. If you use
>> host stretch and build for wheezy, there will be big gap in versions.
> 
> Thanks for pointing that out. That is a mistake just like the ARCH
> thing Claudius pointed out. I will look into that.
> I kindly ask you to review the patch again and provide such feedback
> inline.

Sure, will do.

>>>    
>>>> 2. Can we just add optional task to generic pipeline which
>>>> generates debian folder? Or make it as a part of do_build: if [ !
>>>> -d ${D}/debian ]; then blablabla.
>>>
>>> That would be adding support for 'debian'-izing a source package.
>>>
>>> This patch is more about packaging binaries and creating
>>> meta-packages.
>>
>> Ok.
>>
>