From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6785506002995249152 X-Received: by 2002:aa7:c811:: with SMTP id a17mr2619034edt.272.1579873862554; Fri, 24 Jan 2020 05:51:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:1d30:: with SMTP id dh16ls679995edb.10.gmail; Fri, 24 Jan 2020 05:51:01 -0800 (PST) X-Google-Smtp-Source: APXvYqxLCuAc6YUxE0BGO6XzuWoVlDg6ObHjd+gZNuVcQGVId/JxRv0GL2xp9FtPOumHnRuoJvpe X-Received: by 2002:aa7:c3c2:: with SMTP id l2mr1711785edr.120.1579873861856; Fri, 24 Jan 2020 05:51:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1579873861; cv=none; d=google.com; s=arc-20160816; b=GbNN46PIaA0mYkDidZiM0ei5T65QUbvV8zayNJCycoDjNFuuSgTtaRzoJJQcYwQZ9G coG54aGgUwtZaeJqMl/wzFc0dFrP4SwmtYTyzZ+Uvxb2s5+d9Cwtec1Ndw2yi9TVQRmf aIHM0F35NeHIW7fx5VvEy65Ts7sQO8bO3fjjpDGrB+yZys+c4BNZymNYE7crTr3pa0NA 3qdO10L7Jka7/A7fOR+DW5YhcP8ExFjSM8/0xOBwf8U4wQpdcq6SZvUWKPT47tMCcNy5 XF5ev7ntfYYeTkGk/LyoJk8vucXYjRx3ZTt/UkP7h2igBokSri98UxZR4vfGG/XqLQpa bdhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:to:from:date; bh=Uq1rt2/6tFLx7YwiAZFpDRdpMcQTVzBTjMYXvO5MzQI=; b=LoLQcaeTPrCJRMVcyMBAhf6VzjnPZOCWjQGuULYSu4yGCydOsOO7HwWJEaG3S6rhDl YTYg3HTfOhtgjVxeD+H9zu+Y80ffIId0iTDivK9TR7xVYvnjhZgbReB0FHdPpSdF0C3/ OxWSB2En9gchjimnVKN/n90lExJIgnxVTOAeZNSHRx+lElWA2j7omzo4ftekW9BosjA7 dNDx+iubCF53dqUUgdUm75lrGXVCkN5i73Ou4WrzbQcfLMYfpmtmw/+hg1xjTlWd4Scq q5U0oaZIOkavlygchf/XoezM3FG5yh2UqUU8g+dPugrEsg4BrqpgSzvANil9Nwuru5bZ mIjQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id a5si71155edx.1.2020.01.24.05.51.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Jan 2020 05:51:01 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 00ODp1ur010201 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 24 Jan 2020 14:51:01 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 00ODp1Ke032431 for ; Fri, 24 Jan 2020 14:51:01 +0100 Date: Fri, 24 Jan 2020 14:51:00 +0100 From: Henning Schild To: Subject: Re: [PATCH] lib/oe/path: try hardlinking instead of guessing when it might fail Message-ID: <20200124145100.3188783f@md1za8fc.ad001.siemens.net> In-Reply-To: <20200124134847.5052-1-henning.schild@siemens.com> References: <20200124134847.5052-1-henning.schild@siemens.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: E9bTfuIsspqm This is related to fixing an issue with "--exclude-path" in wic. On the Isar side i have everything in place and might send it. But we need that upstream fix ... or some workaround if that does not get merged. Henning On Fri, 24 Jan 2020 14:48:47 +0100 Henning Schild wrote: > From: Henning Schild > > The comparison of the stat st_dev is not enough to judge whether > hardlinking will work. One example would be where you try and hardlink > across two bind-mounts of a directory. The st_dev will be the same and > the operation will still fail. > > Instead of implementing a check to try and figure out hardlink support > just try hardlinking and fall back to a copy when running into an > exception. > > Signed-off-by: Henning Schild > --- > meta/lib/oe/path.py | 24 ++++++++++++++++++------ > 1 file changed, 18 insertions(+), 6 deletions(-) > > diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py > index fa209b9795..082972457b 100644 > --- a/meta/lib/oe/path.py > +++ b/meta/lib/oe/path.py > @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): > if os.path.isdir(src) and not len(os.listdir(src)): > return > > - if (os.stat(src).st_dev == os.stat(dst).st_dev): > + canhard = False > + testfile = None > + for root, dirs, files in os.walk(src): > + if len(files): > + testfile = os.path.join(root, files[0]) > + break > + > + if testfile is not None: > + try: > + os.link(testfile, os.path.join(dst, 'testfile')) > + os.unlink(os.path.join(dst, 'testfile')) > + canhard = True > + except Exception as e: > + bb.debug(2, "Hardlink test failed with " + str(e)) > + > + if (canhard): > # Need to copy directories only with tar first since cp will > error if two # writers try and create a directory at the same time > cmd = "cd %s; find . -type d -print | tar --xattrs > --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - > | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) > @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def > copyhardlink(src, dst): """Make a hard link when possible, otherwise > copy.""" > - # We need to stat the destination directory as the destination > file probably > - # doesn't exist yet. > - dstdir = os.path.dirname(dst) > - if os.stat(src).st_dev == os.stat(dstdir).st_dev: > + try: > os.link(src, dst) > - else: > + except OSError: > shutil.copy(src, dst) > > def remove(path, recurse=True):