From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6785506002995249152 X-Received: by 2002:a7b:c1c2:: with SMTP id a2mr3432202wmj.52.1579873730201; Fri, 24 Jan 2020 05:48:50 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ea84:: with SMTP id s4ls1600962wrm.11.gmail; Fri, 24 Jan 2020 05:48:49 -0800 (PST) X-Google-Smtp-Source: APXvYqy/8DCR1rT9qNhzFKBKuwxJs9zK8h9Jacv4qUVybwikUXAWULKA2xfRCCfJWyeXlvxJyOHB X-Received: by 2002:a5d:428c:: with SMTP id k12mr4734133wrq.57.1579873729453; Fri, 24 Jan 2020 05:48:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1579873729; cv=none; d=google.com; s=arc-20160816; b=kOs0ik7C+YJ8sH1V37gNLFXLqnCikS1UGQZfzjxZxQ+V9IjCOMW2H9yjZvx5FmnZsW Ixs5Ogl6g/8c6rqJox7itZgLddBjF91CF3EOZFw0Sl3FkJ5BjSbIyVgwkSJ478+lLLXd aKQRqi/l+3tRPpoOi+Yv1lNzQ+8D8L3g6bPrvf2iq6VxSkvKBgmASkyUZ2oVikJ8rPHG wnomxf3BCSg5Q3ibUteQhB8ae/q8xeMU3qBKstsIrERsd27eR+8sqw81yFPKZV32i3c9 ktUeIrb8gnGSfpUhe4xYYYNMunS+/55cIXOdRmENvsfhmpCkn0Ds0xlce9KUsljNO8vS tYjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=JdfOhOisLBXVdhO/lVgDLfE1j3kH7MVin5CUROUsHlM=; b=Y+cWqrtFVIaDHJAMLUaxhG2+QY56BR2+5vpHJGQv6mf53yfueIl45WkxovAGBnbDcT P2mB8wcI2RGeIh2DdmUR2SFUfd4OWTiEz/g9gVlkPE+2VIF+f5745In10DJbPn7eZusG GSNib6Ke8Xq6BGFzO5ZfZxJLCBhJfFJdXaIGgRJhVSX3EV6L+7gMk/lDRoMYIMHMAlRw B0n9CDGgg/A9heG10/QtlK1dN4KvOkOlW2d8Aeud2IKVGNnUttAleIEcXgqen+WQkep4 0IMIW5Egz3zVOOQm82/QiTd2Mtu6ntw/Sq6KqqUVKYVHrm3mnooqRg6tOcl2IkrGex/m LkxA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id m12si278887wrq.1.2020.01.24.05.48.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Jan 2020 05:48:49 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 00ODmn0b030741 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Jan 2020 14:48:49 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 00ODmm5o018049; Fri, 24 Jan 2020 14:48:48 +0100 From: Henning Schild To: openembedded-core@lists.openembedded.org Cc: isar-users@googlegroups.com, Henning Schild Subject: [PATCH] lib/oe/path: try hardlinking instead of guessing when it might fail Date: Fri, 24 Jan 2020 14:48:47 +0100 Message-Id: <20200124134847.5052-1-henning.schild@siemens.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: YnXD3bUf17rC From: Henning Schild The comparison of the stat st_dev is not enough to judge whether hardlinking will work. One example would be where you try and hardlink across two bind-mounts of a directory. The st_dev will be the same and the operation will still fail. Instead of implementing a check to try and figure out hardlink support just try hardlinking and fall back to a copy when running into an exception. Signed-off-by: Henning Schild --- meta/lib/oe/path.py | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py index fa209b9795..082972457b 100644 --- a/meta/lib/oe/path.py +++ b/meta/lib/oe/path.py @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): if os.path.isdir(src) and not len(os.listdir(src)): return - if (os.stat(src).st_dev == os.stat(dst).st_dev): + canhard = False + testfile = None + for root, dirs, files in os.walk(src): + if len(files): + testfile = os.path.join(root, files[0]) + break + + if testfile is not None: + try: + os.link(testfile, os.path.join(dst, 'testfile')) + os.unlink(os.path.join(dst, 'testfile')) + canhard = True + except Exception as e: + bb.debug(2, "Hardlink test failed with " + str(e)) + + if (canhard): # Need to copy directories only with tar first since cp will error if two # writers try and create a directory at the same time cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def copyhardlink(src, dst): """Make a hard link when possible, otherwise copy.""" - # We need to stat the destination directory as the destination file probably - # doesn't exist yet. - dstdir = os.path.dirname(dst) - if os.stat(src).st_dev == os.stat(dstdir).st_dev: + try: os.link(src, dst) - else: + except OSError: shutil.copy(src, dst) def remove(path, recurse=True): -- 2.24.1