From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6864480010557718528 X-Received: by 2002:adf:f8c7:: with SMTP id f7mr5684989wrq.117.1598266770963; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:2ca3:: with SMTP id h3ls4201102wmc.1.canary-gmail; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz1Ip9X1SVZ0dO/l+Fh9SJ0Tt8XiZmur7XOrZKkurWJrIyCKvOuq+bRniHDDoLHtjIfc1Vz X-Received: by 2002:a1c:dc45:: with SMTP id t66mr5377622wmg.17.1598266770282; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1598266770; cv=none; d=google.com; s=arc-20160816; b=IZ1qivvRthWJvR99Xgefj6pK5O/iJG0mXGt4pORmvMrsfxUCdk+BssrJK8x7GYVYVl 1Wpx+1uPsJAGPkpN/obsJOhIbMVvjMmK384+yinxuI1NxAUTTGepm+Ny+hZ0cX81CWuR u+d4lU8Oi9zCo427Fh2FpbgFNFFK8EOu4uGu+BzzBgfm9CIYJ7tO91MkvRSS9nK4gnNP KlucqUoShzFskzkQzeHzIY5Z+j1gvyRRGfMDB35AgH3rcNjoDdvFiWDnUfJNJZcgu9bj 8wXgLuoaLm6mSL+9HrJjfpiALL9056A9+NAYazT58g0OFjE+CsV2z7J+u5HL3uGK4NxF pYMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:to:from :ironport-sdr:ironport-sdr; bh=3hESc+kJKtwqKrBuuST3QyiQQb+6pEbodWyFniokofA=; b=G9a5sSfrRY/Bl3T33hpdX8khXO9heXML9vgF70A7C98tbAkGBexYFv0F7GNqNl4EPO q1hIUvYJYK/6K1P1cpC/DYc2cO51EMsrgLoeayZL98HqkyDL8pTQcVQ8C/cFJTgoySi8 5OskPZgTYD+iCKz9OW3/9j5AXx0ytDuXoDJ+EFa1S7cuUDJlGvzLHE+EBnMlSlO+MJgA M1r0pWiG0Vre8yWDWNy4BUemj2oZNknKIrQ/S3YC0GyQgGepmrheb+oA0Pb52SFFwjma 8n7VkvwqlzK4lUrOLQYUoQNSBXsaVnWOxBaTybPs3bM36Yo4I9psaHMLsfh4aOXWa1MO M76w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Return-Path: Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com. [68.232.137.252]) by gmr-mx.google.com with ESMTPS id i11si331160wra.3.2020.08.24.03.59.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Aug 2020 03:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) client-ip=68.232.137.252; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com IronPort-SDR: tTdPFp6Lgy7WMELVL0LwhMjauhf85qs+viCChWkNQz3iV8kQYnGVfttgkVKJHtz79DQdz0tkdS MdD6nuJnUmfbdj0bosrFTLYnSAWXTMyASVskAeOr/iDDLl6Lk2kkTuAPYL5JCs6BkpkjVLS8XB nKxjFifNRIi868iD9llLuMZme2TjoMiMSsrNOupzKYOwdB1fSudGwRe3d2+4D5V4XpL5jpfAje shj4VE85HoAR6jCPyfFhrxAXwfAtzQIXT/fGEQdpALx8e5jiK15fxIttm0/ckdaGjRMOi3stcC 25c= X-IronPort-AV: E=Sophos;i="5.76,348,1592899200"; d="scan'208";a="52309707" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa4.mentor.iphmx.com with ESMTP; 24 Aug 2020 02:59:28 -0800 IronPort-SDR: yQ2RtRZmQg4UncDX3/gnC17Y1tFD67v67zEaui3Rn6L7RVEglD6O9DWnfYZDC24CFQ90Z3qaQd Gn7B8dfywwCVFBtKHAN2om/c/e6IHiJ6K01xckfLOEfTfPMDJABMB+gFm0MqzzBgfU25J2MJ8f RT1kjzfzTvDGCAwWkLMwL5uec8NLj78FmbvGAX1FirvwkXz9f5PZiW0Tq/IABjZq406aPGXTul Y3xG/wWMBefoSJc9Fxurje0BYjjf6KEs5AjL6xjG5YTTC0Flx+r9IY7Z9aB/wu9tLyNlVTvSsF Fr8= From: Vijai Kumar K To: , Subject: [RFC PATCH 05/10] lib/oe/path: try hardlinking instead of guessing when it might fail Date: Mon, 24 Aug 2020 16:29:00 +0530 Message-ID: <20200824105905.21110-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200824092750.19642-5-Vijaikumar_Kanagarajan@mentor.com> References: <20200824092750.19642-5-Vijaikumar_Kanagarajan@mentor.com> MIME-Version: 1.0 Content-Type: text/plain Return-Path: Vijaikumar_Kanagarajan@mentor.com X-ClientProxiedBy: svr-orw-mbx-08.mgc.mentorg.com (147.34.90.208) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-TUID: Mp+F+VwjZBb4 From: Henning Schild The comparison of the stat st_dev is not enough to judge whether hardlinking will work. One example would be where you try and hardlink across two bind-mounts of a directory. The st_dev will be the same and the operation will still fail. Instead of implementing a check to try and figure out hardlink support just try hardlinking and fall back to a copy when running into an exception. Signed-off-by: Henning Schild Signed-off-by: Richard Purdie [Vijai: Imported from OE core f5571bda8327f927feb23b167ab4594b7d0c95bc] Signed-off-by: Vijai Kumar K --- meta/lib/oe/path.py | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py index c6bb604..3506e2c 100644 --- a/meta/lib/oe/path.py +++ b/meta/lib/oe/path.py @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): if os.path.isdir(src) and not len(os.listdir(src)): return - if (os.stat(src).st_dev == os.stat(dst).st_dev): + canhard = False + testfile = None + for root, dirs, files in os.walk(src): + if len(files): + testfile = os.path.join(root, files[0]) + break + + if testfile is not None: + try: + os.link(testfile, os.path.join(dst, 'testfile')) + os.unlink(os.path.join(dst, 'testfile')) + canhard = True + except Exception as e: + bb.debug(2, "Hardlink test failed with " + str(e)) + + if (canhard): # Need to copy directories only with tar first since cp will error if two # writers try and create a directory at the same time cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def copyhardlink(src, dst): """Make a hard link when possible, otherwise copy.""" - # We need to stat the destination directory as the destination file probably - # doesn't exist yet. - dstdir = os.path.dirname(dst) - if os.stat(src).st_dev == os.stat(dstdir).st_dev: + try: os.link(src, dst) - else: + except OSError: shutil.copy(src, dst) def remove(path, recurse=True): -- 2.17.1