Re: do_adjust_git in dpkg-base.bbclass not working for gitsm fetcher

[Jan Kiszka <jan.kiszka@siemens.com>]

On 01.12.20 16:40, Henning Schild wrote:
> Am Tue, 1 Dec 2020 12:41:08 +0100
> schrieb "[ext] Jan Kiszka" <jan.kiszka@siemens.com>:
> 
>> On 01.12.20 11:41, [ext] Raphael Lisicki wrote:
>>> Hi,
>>>
>>> the function do_adjust_git does not work if the fetcher has been
>>> set to "gitsm" which is required if the project contains git
>>> submodules.
>>>
>>> See
>>> https://github.com/siemens/isar/blob/master/meta/classes/dpkg-base.bbclass#L26
>>>
>>>
>>> This will cause the .git/objects/info/alternates remain unmodified,
>>> which in turn breaks git from inside the buildchroot.  
>>
>> From
>> https://www.yoctoproject.org/docs/latest/bitbake-user-manual/bitbake-user-manual.html#gitsm-fetcher:
>>
>> "The Git Submodules fetcher is not a complete fetcher implementation.
>> The fetcher has known issues where it does not use the normal source
>> mirroring infrastructure properly. [...]"
>>
>> For that reason already, I would strongly discourage the use of this
>> fetcher. And that's why it was never tested with Isar (to my best
>> knowledge).
> 
> I have seen people use git submodules in isar recipes. But they are
> hairy indeed. A big issue is whether those submodules are absolute or
> relative to their main repo. Relative is your best bet, absolute will
> break authentication since you might find yourself wanting to clone with
> ssh and gitsm contains absolute https links.
> 
> If you control that main repo, you might want to switch to relative ...
> if you can. Doing that breaks the "forking workflow" ... now every
> contributor needs to fork all, or switch to absolute before working.
> 
> The worst case would be nested submodules ...
> 
> I found it best to switch over to repeating the submodule stuff in the
> recipe, naming all modules with their shas and where they should be
> cloned. Hard to maintain ... but you can write helper scripts writing
> ".inc" files containing the bits and using a clone with initialized
> modules as input.
> 
> SRC_URI = "gitsm://foo.git;destsuffix=${P}"
> SRCREV = "0815"
> 
> becomes
> 
> SRC_URI = "git://foo.git;destsuffix=${P};name=main
> SRCREV_main = "0815"
> SRC_URI += "git://bar.git;destsuffix=${P}/modules/bar;name=module1
> SRC_URI += "git://baz.git;destsuffix=${P}/mod/baz;name=module2
> SRCREV_module1 = "1234"
> SRCREV_module2 = "abcd"
> 
> Those last 4 lines can be generated from ".gitmodules" and "git ls-tree
> <main-repo-sha> <sm-path>" and stored in an include.
> 

That's a better pattern, indeed.

Maybe someone wants to hack up some scripts/unroll-gitsm.sh to spit out
those lines for a given recipe.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux