From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6982479695616933888 X-Received: by 2002:a25:3213:: with SMTP id y19mr36864287yby.92.1625735241054; Thu, 08 Jul 2021 02:07:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5b:305:: with SMTP id j5ls882163ybp.3.gmail; Thu, 08 Jul 2021 02:07:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyegG8YLAG8qQiyNOFIEfa+/B5RV9FyU/RVsEaaD/tgLXG7Qp5Hi3MdnJXDh+38WydPtwUh X-Received: by 2002:a25:2e0c:: with SMTP id u12mr38525668ybu.491.1625735240642; Thu, 08 Jul 2021 02:07:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625735240; cv=none; d=google.com; s=arc-20160816; b=YhK0QMcA491pK4GaRRy5kxnjdL4hHf6zD/1s3nSTbFlTm6o+NN5BiXjF6SXMebGzuv /F0FkFpbzWsOurfEwbc/v4lv5jMF6ocG6HptpfegNnPRyrk0E247WlaXLnMeQ/2CYBJL +2agKBPKkntVANYsdznZ2Y0/pvQOuvC54naep3qBE/Cmsak3eijNqMQIGCtutdCKjSRZ +sUwkqiEaZqBfpd9nypNpM+9NGobwzgIxoK1dd1UlnteJ9IE2rK5KYb6Au2xH50ZuqY4 MgHWL3z1Jy2yqlxBQ9YdefUb0PhWtVipWhhWdjvqnES6DcgkW9jW1IYR1XmbPgoNwvu4 0bOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:subject:from:to:ironport-sdr:ironport-sdr; bh=LCPyByja9LWN4yQD55dVW1rzwNQbZiFRbKvXCFKrI5c=; b=SHVaQ4UAmUX5U7J6MNhGDv2On48TejJ1IYAvmJ94Q4s/s+2LemX7oWCMpGRhtCgNkI wyU3nRb63K92Yt9OMjPngeuyW4G9VBx9G3d/aDR9lSQcjOLhd6iS9wgk9s33ZPy3Fu+d A0yIbaaBSnhL03Tg77FcDZ8dSFy5SDaeocl20CD2xZ4xZSWl079ahb6CxjBdKVBA94xR It0JBke4h0lUi/2FkJL5HzYcSAS4nbUEhIhaOYYHZhT1uDZfUGdF4zvhfkRkEjhgc+j/ uzwb6gFcIVe3m7TMe3IIH0XuJaEGFLgvwUiEjJiGEUaC/DPKI1naf+wM51CgMX2X9f5m efFw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.141.98 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com Return-Path: Received: from esa2.mentor.iphmx.com (esa2.mentor.iphmx.com. [68.232.141.98]) by gmr-mx.google.com with ESMTPS id s17si202252ybk.2.2021.07.08.02.07.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jul 2021 02:07:20 -0700 (PDT) Received-SPF: pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.141.98 as permitted sender) client-ip=68.232.141.98; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.141.98 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com IronPort-SDR: UgymVhZrAzBfqxsVfZBLdu/X000Jp16v8/TIkRhuTO8takjUjnBQ4FAPQMLXQjkDhMqpjI9ywp ui95In4J9W6vKsxRrZPc8UypsIpjIYWqKNxK5LSE+RITYFiIE7RQEtIjdPqFmKv3wG1ZF4pE3g Dj3pR2xUUv5uO9zhrQoymRHfqSTNiapxlB11Y1sedbf2IbxHYs2VKFmwK1PDRRH+JIuC9WaDVG YmoWkGO0UoZwiTbsh/tj4npX2TEAq8XWTejtCOTvN4K0HqqIo8945vOqMtf2mH+iNYNU+XWa4x uIk= X-IronPort-AV: E=Sophos;i="5.84,222,1620720000"; d="scan'208";a="63290244" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa2.mentor.iphmx.com with ESMTP; 08 Jul 2021 01:07:20 -0800 IronPort-SDR: ZHOZh9ZiqdLUsEsu0rrmGBD3qtls5+E8cAZQvwQMCLDnKcX8dkBdMFdbAyaoyE6pNyYYaoiPGU G5J/n1NaZX9cxHXeSvPmF4bw7o9bDy0T1LfWQtwmNeughuRpuQGzFUDuh/vN1vEPF60u2Smev/ y1vPBq7/b44oHKHKfaCj6aZ4dw9QP910kvV1Ek3FZdltO4V/B7NR1o6sN1g1l0SuzOq6sXbqfG LEVKJI4bBrPLAxkLlWJu+Ag6bZYrqrpOswZRcowWzedqQo+ygt9REyI9/aJjNbiCEH1bweodD+ +GI= To: isar-users From: Cedric Hombourger Subject: [RFC] using lightweight containers instead of chroot CC: "MacDonald, Joe" Message-ID: <11b6ea24-b31a-a417-bcd9-0b32c5abe308@mentor.com> Date: Thu, 8 Jul 2021 11:07:14 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Return-Path: cedric_hombourger@mentor.com X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-09.mgc.mentorg.com (139.181.222.9) To SVR-IES-MBX-03.mgc.mentorg.com (139.181.222.3) X-TUID: FIMgzb4ykXld Hello, This is a RFC for us to replace uses of chroot within the Isar classes to use Linux namespaces. Motivation We will be submitting another RFC for per-recipe buildchroots but we should note that it would likely increase the number of active bind mounts during our builds. So before we discuss that, we may want to discuss how we run our build scripts today using chroots. There are numerous challenges with our current implementation and attempts to , Examples would include: 1e31d5a events: Warn if mounted paths left d21d495 dpkg: Make mount buildroot reliable 22c42de dpkg-base: Warn about unmounting problems ... And you have found yourself in the situation where Isar wiped out /dev or your layers (at least I did) chroot is a powerful tool but it is starting to show its age and we see how much burden is on us to setup and shutdown our chroot environment Proposal We may want to use unshare(1) to create a mount namespace where we will create our bind mounts, chroot into the buildchroot and run the specified command/script The immediate benefit of this approach is that all mounts automatically disappear as the supplied command exits (whether it aborts prematurely because of an error or normally on completion). Another nice benefit is that bind mounts we created within this namespace are not (directly) visible from the parent namespace However, we found that running scripts within an unshare environment may not be as easy as chroot. We would welcome feedback on the code snippets provided below if you happen to have some better ideas. def isar_user_spec():     import os     return '%d:%d' % (os.getuid(), os.getgid()) ISAR_USER_SPEC    = "${@ isar_user_spec() }" ISAR_UNSHARE_CMD  = "sudo unshare --pid --fork --ipc --mount sh -ex" ISAR_CHROOT_SHELL = "sh -ex" ISAR_CHROOT_ROOT  = "chroot ${BUILDCHROOT_DIR} ${ISAR_CHROOT_SHELL}" ISAR_CHROOT_USER  = "chroot --userspec='${ISAR_USER_SPEC}' ${BUILDCHROOT_DIR} ${ISAR_CHROOT_SHELL}" # Would be similar to buildchroot_do_mounts but will happen in a separate mount namespace BUILDCHROOT_DO_MOUNTS = "                                                         \     mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${BUILDCHROOT_DIR}/isar-apt'     ; \     mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'                     ; \     mount -t proc none '${BUILDCHROOT_DIR}/proc'                                ; \     mount --rbind /sys '${BUILDCHROOT_DIR}/sys'                                 ; \     mount -t tmpfs -o rw,nosuid,nodev,seclabel none ${BUILDCHROOT_DIR}/dev/shm  ; \     mount -o bind /dev/pts ${BUILDCHROOT_DIR}/dev/pts                             \ " # Would be similar to dpkg_do_mounts but will happen in a separate mount namespace DPKG_DO_MOUNTS = "                         \     ${BUILDCHROOT_DO_MOUNTS}             ; \     mkdir -p ${BUILDROOT}                ; \     mount --bind ${WORKDIR} ${BUILDROOT}   \ " # Build package from sources using build script _runbuild() {     export arch=${1}     E="${@ isar_export_proxies(d)}"     (   cat <<"        UNSHARE"             ${DPKG_DO_MOUNTS}             (   cat <<"                SCRIPT"                     export DEB_BUILD_OPTIONS="${DEB_BUILD_OPTIONS}"                     export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}"                     export PARALLEL_MAKE="${PARALLEL_MAKE}"                     /isar/build.sh ${PP}/${PPS} ${arch}                 SCRIPT             ) | ${ISAR_CHROOT_USER}         UNSHARE     ) | envsubst '$arch' | ${ISAR_UNSHARE_CMD} } dpkg_runbuild() {     ( _runbuild ${PACKAGE_ARCH} ) } PS: I am not very happy with the need to feed the script to execute under unshare     via stdin, if there are better ways, we would be happy to consider them! Proposed Next steps 1. Collect feedback and answer questions    a. is the use of unshare a good idea? (no is an OK answer!)    b. can we come up with a better code construct? 2. Check compatibility with containerized-builds (e.g. from within a kas build) 3. Check for better ways to spawn scripts 4. Implement and submit RFC patches Future RFCs - per-recipe buildchroots (mimic sbuild) Cedric