From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6462206452988641280 X-Received: by 10.25.22.160 with SMTP id 32mr318796lfw.8.1504609800826; Tue, 05 Sep 2017 04:10:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.92.204 with SMTP id q195ls1738462wmb.16.canary-gmail; Tue, 05 Sep 2017 04:10:00 -0700 (PDT) X-Google-Smtp-Source: ADKCNb6djLsjZfHgXpvjRlg20C8LxONKK+6KDonOEk/5VI6772UJBjhvrzeEeOJB4rQNyYE5aD3B X-Received: by 10.28.29.215 with SMTP id d206mr341942wmd.7.1504609799975; Tue, 05 Sep 2017 04:09:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504609799; cv=none; d=google.com; s=arc-20160816; b=1HaMIX0cjjFL4BG6LXjg2EJwJPTThaW7aNyzh9we0YrIAHxFm8U180WcKbkSAX/edL OprZ1ht8DZG+MwFeStpXETwNnZJZbPMQwWWSBKr0GXB97qPBGx1kfVtWyjKwKxMIbSGj 9FjGxA/mv7VHbzeYo9dSXa3Pqkb8lOd4VEeKNNJDfxU/pIwvLmDs4oMAJutK3t+11ryp qdG11ZtcO3AmNvfN1e0bCobTOrRyBa6xYZRjyrPXNfPtINqFcXbQ5AFtv8jUP1hEudFS BcNhn7/9TnDf6uk5tJ6LYHeZ5ADLudyu4llO6ly28MAlSNJWLv/iZh6IyKE4QcrJXBOG 4jcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=EXTlsDCZCn+VL7dCn2/LjtDw2Arm6y+z6sa/Nsyzgmg=; b=qr5ilHkKnRLUvpC4IpsDZX112aQLbAjYDfPXQPh2UtghXsMOGQ33poyLfXOiKy+8T9 uDqbpTzWTWERW6tZqBCtBJSO+1kuvUGRGdq1ZyHl3ewttJdl4PD/wSs3t/Cr0GuDb72M ZXQ8Pb/Jg1e++PtFfAjSyZXvQ/jguqEjuvPYSaivSs2m3G+VyR8/kb6dQmB60nYNfdCW 9LgnbtQh2hnQdCnV/FxlTsOc+GHwLbLYI4aAmwpTYFYAeDeYBzASdQ2nTVnkrbMqyoGI /bblDMbcma96huTSqiupAA9+tDiO+uqhKcFU4Pme025WAdGKGNU8ASQCPFqnF0fCob5t ivGg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id 200si47579wmj.0.2017.09.05.04.09.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Sep 2017 04:09:59 -0700 (PDT) Received-SPF: neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id v85B9xpL000634 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 5 Sep 2017 13:09:59 +0200 Received: from [139.25.68.223] (linux-ses-ext02.ppmd.siemens.net [139.25.68.223]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id v85B9xcH007122; Tue, 5 Sep 2017 13:09:59 +0200 Subject: Re: Isar build tree structure To: Alexander Smirnov Cc: isar-users@googlegroups.com, "johann.pfefferl@siemens.com >> PFEFFERL JOHANN Z003CRBA" References: <7c829771-d78b-7409-ce86-74fae0f15dc4@ilbers.de> <2b9b891a-055a-2e4f-60ba-dc87972769d9@siemens.com> <44061e32-aa48-b432-ead5-5d6627b707b4@ilbers.de> <1fb116bd-d3a6-bca9-9105-cc32b0ba276d@ilbers.de> From: Claudius Heine Message-ID: <8aa6847a-63ba-49ec-32ec-b77a38800ac8@siemens.com> Date: Tue, 5 Sep 2017 13:09:59 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <1fb116bd-d3a6-bca9-9105-cc32b0ba276d@ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 2TGLy7RmCdA/ Hi, On 09/05/2017 11:35 AM, Alexander Smirnov wrote: > > > On 09/05/2017 12:28 PM, Claudius Heine wrote: >> On 09/05/2017 11:13 AM, Alexander Smirnov wrote: >>> >>> >>> On 09/05/2017 12:06 PM, Claudius Heine wrote: >>>> Hi, >>>> >>>> On 09/05/2017 10:21 AM, Alexander Smirnov wrote: >>>>> Hi all, >>>>> >>>>> I'd like to discuss build folders tree to somehow chose more >>>>> suitable approach. >>>>> >>>>> 1. How it's done at the moment. >>>>> >>>>> The folder tree contains the following main paths: >>>>> >>>>> - tmp/work/${PP} >>>>> >>>>> It's a $WORKDIR for each recipe, here is stored bibtbake logs and >>>>> commands, and data for packages that doesn't require buildchroot >>>>> (for example, buildchroot and isar-image-base). >>>>> >>>>> - tmp/work/buildchroot/${DISTRO}-${ARCH}/rootfs/home/build/${PN} >>>>> >>>>> This is the $BUILDROOT for packages, that require compilation in >>>>> chroot. It is used as destination folder for unpack task. >>>>> >>>>> Benefits: >>>>> >>>>> - Maximal common OE-like folder structure for all kind of >>>>> recipes. You can find the information about specific package in >>>>> 'tmp/work/${PP}'. And it doesn't matter if this package uses dpkg >>>>> class, or not. >>>>> >>>>> - After buildchroot cleanup, the build information is not loosed. >>>>> >>>>> - For multiple buildchroots in project, there will be single >>>>> place for package meta information. >>>>> >>>>> 2. Remove $BUILDROOT. >>>>> >>>>> This would mean that $WORKDIR location may vary depending on recipe >>>>> type. So the recipes data will be distributed across buildchroots >>>>> and tmp/work dir. >>>>> >>>>> Benefits: >>>>> >>>>> - Each recipe will use single folder for everything. >>>>> >>>>> >>>>> >>>>> So... Ideas and opinions? :-) >>>>> >>>> >>>> Why not both benefits? Using bind mount to put the workdir of >>>> recipes into the buildchroot? >>> >>> Mount is very difficult to manage. For example if something failed >>> during build, it's hard to check in next build what is already >>> mounted and what is not yet. >>> >>> But your idea is good. Could we use symlinks instead? >> >> If you are in a chroot environment the paths of the symlinks are wrong >> since they reference a different root directory. > > Yeah, you are right... So probably mount could be an option: > > = do_unpack = > 1. unpack to WORKDIR (tmp/work/${PP}) > > = do_build = > 1. mount respective folder in buildchroot > 2. build > 3. unmount respective folder in buildchroot > > This approach will be quite good compatible with OE. If we go with proot then all this mounting could be done without requiring root or this build step. I could be done like this: $ proot -S ${BUILDROOT} -b ${WORKDIR}:${CHROOT_BUILDDIR} build.sh One problem with proot is this: $ mkdir test $ cd test $ proot -0 # id uid=0(root) gid=0(root) groups=0(root) # touch testfile # ls -al total 8 drwxr-xr-x 2 root root 4096 Sep 5 13:02 . drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. -rw-r--r-- 1 root root 0 Sep 5 13:02 testfile # chown 123:123 testfile # ls -al total 8 drwxr-xr-x 2 root root 4096 Sep 5 13:02 . drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. -rw-r--r-- 1 root root 0 Sep 5 13:02 testfile # exit $ ls -al total 8 drwxr-xr-x 2 user user 4096 Sep 5 13:02 . drwxrwxrwt 18 root root 4096 Sep 5 13:02 .. -rw-r--r-- 1 user user 0 Sep 5 13:02 testfile It maps just everything from the current user to root and back. So multiple users aren't supported here. But maybe this can be patched. >>>> >>>> Maybe we should take a look at proot [1] and use this instead of >>>> chroot, because its more flexible, does not require privileges, etc. >>>> >>>> [1] https://github.com/proot-me/PRoot/blob/master/doc/proot/manual.txt >>>> >>> >>> Do you know if it fully supports dpkg & Co? >> >> I currently use proot in some parts of my isar fork already. And from >> my experience it works rather well. Since it uses the ptrace >> systemcall, static binaries work work as well. >> > > Do you have patch for this, could you please send it as RFC? I don't have anything worth submitting (even as RFC) concerning proot. But AFAIK Johannes has converted more of isar to use proot. Maybe he can give some input for this. Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de