From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6462206452988641280 X-Received: by 10.28.29.215 with SMTP id d206mr220266wmd.7.1504707435575; Wed, 06 Sep 2017 07:17:15 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.22.15 with SMTP id w15ls268660ljd.8.gmail; Wed, 06 Sep 2017 07:17:15 -0700 (PDT) X-Google-Smtp-Source: ADKCNb6fxgMMPfqrllA/uJ1BSMIpVbC0iqYSUEjQ+uYAeHe6GmDcbD40ep7o6SnDiVC7f0aNdZHR X-Received: by 10.46.82.12 with SMTP id g12mr234286ljb.24.1504707435083; Wed, 06 Sep 2017 07:17:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504707435; cv=none; d=google.com; s=arc-20160816; b=QCbnmrG1ESKMk85Kryl6XrvqdNX7xgZXYOI+XLPVWkeSr3xvyzub5de+x302NRiMqs XueZuiofQOZb89brJOc5Mb5ePXHw4NwJMemHDOXU3ivQHfK1ZnnkJFunaE7F5xPm1amz 7Bhn9Ly+ZKH/47gFS+6GCz5GqmY1Qx8PZ2W+RbW4b1DIJ7GF1FZB7AW4FMbpwPAdOfuC JjWWExmQzxbDM5QiPHxFlplvaR5fOq2fKJ/uyIa1uDjiTuI3rRXiGD2Msf98DmRn1bH7 NrIm7kXWn3tE/waMVjBC6Bo4CYRGckiJP3zaYQZ/6SskgZBHDTV2DIqJARUPTVswHPNc Pwlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:cc:to:from:subject :arc-authentication-results; bh=il8erl+Z/vspLFLiEhaI5aJSNAFUuCrsF2sp6EFwrIo=; b=boppFd9OZNB3Ltf9XYQERiDHc/wYOFhkE35zs0djY2SYBVBABKoS4/w4+qD5AESVeK 2pw+7fPed1yTl8Pk7UtyoNOWFWe0Ng57Ws687K2yFXd3tO2SS07JqqXUUcCCb/pqve4M jnGA00itcNuuAQy/nltPMKOd4KQm+IhVN+zqNtj1yZxZ7MdW8NmXB7kWWwO+PV0g5SD5 aBCvBsLsoFaQA0MhK74SHQNBX2UQFDUr6QEE9Rlso+bEkvv+8jFVNQXlmRCd8j2n+e9Y V9v8VHU3rYELmWgf2jPg6B0OnNrYTa80MzGY5M+LJA48N6ds5WbpEe/U6UedZSizPHsC 2oJQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id d82si138000wmd.1.2017.09.06.07.17.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Sep 2017 07:17:15 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.14 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id v86EHEF2003992 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Sep 2017 16:17:14 +0200 Received: from [139.25.68.223] (linux-ses-ext02.ppmd.siemens.net [139.25.68.223]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id v86EHEhV016619; Wed, 6 Sep 2017 16:17:14 +0200 Subject: Re: Isar build tree structure From: Claudius Heine To: Alexander Smirnov Cc: isar-users@googlegroups.com, "johann.pfefferl@siemens.com >> PFEFFERL JOHANN Z003CRBA" References: <7c829771-d78b-7409-ce86-74fae0f15dc4@ilbers.de> <2b9b891a-055a-2e4f-60ba-dc87972769d9@siemens.com> <44061e32-aa48-b432-ead5-5d6627b707b4@ilbers.de> <1fb116bd-d3a6-bca9-9105-cc32b0ba276d@ilbers.de> <8aa6847a-63ba-49ec-32ec-b77a38800ac8@siemens.com> Message-ID: <13294528-62f7-cf32-fece-1f89687a188a@siemens.com> Date: Wed, 6 Sep 2017 16:17:14 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <8aa6847a-63ba-49ec-32ec-b77a38800ac8@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: eT2RA0ylAHtI Hi, On 09/05/2017 01:09 PM, [ext] Claudius Heine wrote: > Hi, > > On 09/05/2017 11:35 AM, Alexander Smirnov wrote: >> >> >> On 09/05/2017 12:28 PM, Claudius Heine wrote: >>> On 09/05/2017 11:13 AM, Alexander Smirnov wrote: >>>> >>>> >>>> On 09/05/2017 12:06 PM, Claudius Heine wrote: >>>>> Hi, >>>>> >>>>> On 09/05/2017 10:21 AM, Alexander Smirnov wrote: >>>>>> Hi all, >>>>>> >>>>>> I'd like to discuss build folders tree to somehow chose more >>>>>> suitable approach. >>>>>> >>>>>> 1. How it's done at the moment. >>>>>> >>>>>> The folder tree contains the following main paths: >>>>>> >>>>>> - tmp/work/${PP} >>>>>> >>>>>> It's a $WORKDIR for each recipe, here is stored bibtbake logs and >>>>>> commands, and data for packages that doesn't require buildchroot >>>>>> (for example, buildchroot and isar-image-base). >>>>>> >>>>>> - tmp/work/buildchroot/${DISTRO}-${ARCH}/rootfs/home/build/${PN} >>>>>> >>>>>> This is the $BUILDROOT for packages, that require compilation in >>>>>> chroot. It is used as destination folder for unpack task. >>>>>> >>>>>> Benefits: >>>>>> >>>>>> - Maximal common OE-like folder structure for all kind of >>>>>> recipes. You can find the information about specific package in >>>>>> 'tmp/work/${PP}'. And it doesn't matter if this package uses dpkg >>>>>> class, or not. >>>>>> >>>>>> - After buildchroot cleanup, the build information is not loosed. >>>>>> >>>>>> - For multiple buildchroots in project, there will be single >>>>>> place for package meta information. >>>>>> >>>>>> 2. Remove $BUILDROOT. >>>>>> >>>>>> This would mean that $WORKDIR location may vary depending on >>>>>> recipe type. So the recipes data will be distributed across >>>>>> buildchroots and tmp/work dir. >>>>>> >>>>>> Benefits: >>>>>> >>>>>> - Each recipe will use single folder for everything. >>>>>> >>>>>> >>>>>> >>>>>> So... Ideas and opinions? :-) >>>>>> >>>>> >>>>> Why not both benefits? Using bind mount to put the workdir of >>>>> recipes into the buildchroot? >>>> >>>> Mount is very difficult to manage. For example if something failed >>>> during build, it's hard to check in next build what is already >>>> mounted and what is not yet. >>>> >>>> But your idea is good. Could we use symlinks instead? >>> >>> If you are in a chroot environment the paths of the symlinks are >>> wrong since they reference a different root directory. >> >> Yeah, you are right... So probably mount could be an option: >> >> = do_unpack = >> 1. unpack to WORKDIR (tmp/work/${PP}) >> >> = do_build = >> 1. mount respective folder in buildchroot >> 2. build >> 3. unmount respective folder in buildchroot >> >> This approach will be quite good compatible with OE. > > If we go with proot then all this mounting could be done without > requiring root or this build step. I could be done like this: > > $ proot -S ${BUILDROOT} -b ${WORKDIR}:${CHROOT_BUILDDIR} build.sh > > One problem with proot is this: > > $ mkdir test > $ cd test > $ proot -0 > # id > uid=0(root) gid=0(root) groups=0(root) > # touch testfile > # ls -al > total 8 > drwxr-xr-x 2 root root 4096 Sep 5 13:02 . > drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. > -rw-r--r-- 1 root root 0 Sep 5 13:02 testfile > # chown 123:123 testfile > # ls -al > total 8 > drwxr-xr-x 2 root root 4096 Sep 5 13:02 . > drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. > -rw-r--r-- 1 root root 0 Sep 5 13:02 testfile > # exit > $ ls -al > total 8 > drwxr-xr-x 2 user user 4096 Sep 5 13:02 . > drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. > -rw-r--r-- 1 user user 0 Sep 5 13:02 testfile > > It maps just everything from the current user to root and back. So > multiple users aren't supported here. But maybe this can be patched. I will go a bit deeper for this proposal. The roadmap of proot [1] shows that this point is on their TODO list: * Emulate chown, chmod, and mknod when -0 (fake_id0) is enabled. We have to communicate with the proot dev, maybe they already have an idea how to solve it themselve, but here is my suggestion: Add an maybe two additional parameter to proot, maybe '--file-node-map map.txt' and maybe '--file-node-map-prefix /sysroot'. The 'map.txt' file could have this structure: /path/to/file f 12 13 0664 /path d 12 13 0755 /dev/console c 5 1 0 0 0600 Were the first column is the path with the 'file-node-map-base' as the prefix added internally, the next the type 'f' for file, 'd' for directory and 'c' for character device. If its a device the next two columns are major and minor. the next 3 columns for all node types is the uid gid and permissions. This file is loaded by proot and then it emulates those files. On chown, chmod and mknod the internal struct from proot is changed. When proot exits it saves the internal struct back to 'map.txt'. Since all the files seems to have the correct permission from inside the proot, calling tar from within would create the right permissions inside the resulting tarball. From a correct tarball wic should be able to create the correct image using their tools (pseudo, ...). This way we would not depend on the strangeness of pseudo for our tasks. Cheers, Claudius [1] https://github.com/proot-me/PRoot/blob/afe225dc804e7926e588e8af22fbc5433b773e2d/doc/proot/roadmap.txt -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de