From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6901236327051689984 X-Received: by 2002:a19:7d04:: with SMTP id y4mr1805910lfc.344.1606848692870; Tue, 01 Dec 2020 10:51:32 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:5a02:: with SMTP id q2ls2341570lfn.1.gmail; Tue, 01 Dec 2020 10:51:31 -0800 (PST) X-Google-Smtp-Source: ABdhPJxU4Iy4TwFNDyaqX3bnmEWePdE9QJV/7bVkR/4ot+Tdqe/D5TYJPiL9zrMdV7hPZuO4b8qx X-Received: by 2002:a19:dc55:: with SMTP id f21mr1830960lfj.160.1606848691017; Tue, 01 Dec 2020 10:51:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606848691; cv=none; d=google.com; s=arc-20160816; b=KhVUggFkzkdmWv6PAeLm/GXgDW2qHsKMhZXokLdTV/RN6uD7TrDUfOMjMFHVIalEHK nbfF3qhxwh2FDkoD1+ehBAICEslVXYaETJiSxqcaoa3seQcMvvpgJhuRH/4fsR18i7Vu vXfHOWcmsCGlpdNmdCK6J3xVnmdbnY1ijR8sFQPFQ2ixoYuWNeb6QtdFZP0NDf61JNug 6V2rmdEqU2T4V/q6dxMIUBzd/O1rRq1DiK4G+ntQ8lHhSH3Hu0F/NiQikK4Nif+ehFuE DUjydXSoC3nvV8yuYHM/A+IS6ZKi/akY9ktgSnLOVFYnsPtrwfDWYJGH35TC4P81Q4tG 4w1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=Erhuo6HhZTzxJDNbqFnEf78B14garlrJvexD6F/aXb4=; b=xJvZozdIJGtZLJLJkCeF45RI4g7u+t0cAHKapc4qLcS13ikc3B7HpCh+r04rP8Wz4p dZOgOALKu6DPzNgk8JfNg5D749SmwpSaNU9iSW5XeiMwPqqB//pmcyVz/IPkBzFOb9E4 pTW1wa/9LXL++AdPpnicG1kB2B9u8iDm2sE1kDRnEkL26A0QqWMS+3di3sHVBy8SLQIC 0Z0zyJ9T6QNuouyIkftpXeDL4C4cogmNAafMDooLaKxX6WailMZSJRRlqasI4eDELU4Q NWOdP5h/ifoyXYknLXOe1PIrkY9l/EcUem2ss62ypZhW0bI8//qPGGEFTRLeq07CwyBC dW+Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id f17si20108lfq.9.2020.12.01.10.51.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Dec 2020 10:51:30 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 0B1IpTxd017070 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 1 Dec 2020 19:51:30 +0100 Received: from md1za8fc.ad001.siemens.net ([167.87.54.165]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 0B1IpTEJ018950; Tue, 1 Dec 2020 19:51:29 +0100 Date: Tue, 1 Dec 2020 19:51:28 +0100 From: Henning Schild To: Jan Kiszka Cc: "[ext] Raphael Lisicki" , Subject: Re: do_adjust_git in dpkg-base.bbclass not working for gitsm fetcher Message-ID: <20201201195128.2dea4ee3@md1za8fc.ad001.siemens.net> In-Reply-To: <2f933d46-71cc-c24f-515f-3b94a9634049@siemens.com> References: <20201201164029.00af5bd5@md1za8fc.ad001.siemens.net> <2f933d46-71cc-c24f-515f-3b94a9634049@siemens.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: mcxfEP+DSPrK Am Tue, 1 Dec 2020 17:44:56 +0100 schrieb Jan Kiszka : > On 01.12.20 16:40, Henning Schild wrote: > > Am Tue, 1 Dec 2020 12:41:08 +0100 > > schrieb "[ext] Jan Kiszka" : > > > >> On 01.12.20 11:41, [ext] Raphael Lisicki wrote: > >>> Hi, > >>> > >>> the function do_adjust_git does not work if the fetcher has been > >>> set to "gitsm" which is required if the project contains git > >>> submodules. > >>> > >>> See > >>> https://github.com/siemens/isar/blob/master/meta/classes/dpkg-base.bbclass#L26 > >>> > >>> > >>> This will cause the .git/objects/info/alternates remain > >>> unmodified, which in turn breaks git from inside the buildchroot. > >>> > >> > >> From > >> https://www.yoctoproject.org/docs/latest/bitbake-user-manual/bitbake-user-manual.html#gitsm-fetcher: > >> > >> "The Git Submodules fetcher is not a complete fetcher > >> implementation. The fetcher has known issues where it does not use > >> the normal source mirroring infrastructure properly. [...]" > >> > >> For that reason already, I would strongly discourage the use of > >> this fetcher. And that's why it was never tested with Isar (to my > >> best knowledge). > > > > I have seen people use git submodules in isar recipes. But they are > > hairy indeed. A big issue is whether those submodules are absolute > > or relative to their main repo. Relative is your best bet, absolute > > will break authentication since you might find yourself wanting to > > clone with ssh and gitsm contains absolute https links. > > > > If you control that main repo, you might want to switch to relative > > ... if you can. Doing that breaks the "forking workflow" ... now > > every contributor needs to fork all, or switch to absolute before > > working. > > > > The worst case would be nested submodules ... > > > > I found it best to switch over to repeating the submodule stuff in > > the recipe, naming all modules with their shas and where they > > should be cloned. Hard to maintain ... but you can write helper > > scripts writing ".inc" files containing the bits and using a clone > > with initialized modules as input. > > > > SRC_URI = "gitsm://foo.git;destsuffix=${P}" > > SRCREV = "0815" > > > > becomes > > > > SRC_URI = "git://foo.git;destsuffix=${P};name=main > > SRCREV_main = "0815" > > SRC_URI += "git://bar.git;destsuffix=${P}/modules/bar;name=module1 > > SRC_URI += "git://baz.git;destsuffix=${P}/mod/baz;name=module2 > > SRCREV_module1 = "1234" > > SRCREV_module2 = "abcd" > > > > Those last 4 lines can be generated from ".gitmodules" and "git > > ls-tree " and stored in an include. > > > > That's a better pattern, indeed. > > Maybe someone wants to hack up some scripts/unroll-gitsm.sh to spit > out those lines for a given recipe. Coming up with a truly generic script for that would be like fixing the gitsm fetcher, pretty hard. And if anyone wants to go ahead, fixing the sm fetcher in bitbake would be better. Writing a script for a single recipe is "easy", so that is maybe what people should start with. The real problem is getting getting the url right, it can not be taken from ".gitmodules". Because there you get the ssh vs http(s). Absolute vs. relative ... in the worst case with PREMIRROS. And if you are really unlucky with submodule recursion. I currently work on a layer where we have scripts generating includes to bump recipes. Those scripts are per recipe and very layer-specific. There it is mainly about generating the checksums that bitbake wants in the recipes. Bitbake could learn from gentoo, but that is another story. Henning > Jan >