[vlc-devel] [vlc-commits] contrib: use git-archive and save the hash

[Rémi Denis-Courmont]

On February 21, 2017 3:46:30 AM GMT+02:00, Jean-Baptiste Kempf <jb at videolan.org> wrote:
>This commit breaks the build.
>
>On Mon, 20 Feb 2017, at 21:10, Rémi Denis-Courmont wrote:
>> vlc | branch: master | Rémi Denis-Courmont <remi at remlab.net> | Mon
>Feb 20
>> 22:01:25 2017 +0200| [8fe4168947dd6cabd2aabb84cebc9aa6148e5d43] |
>> committer: Rémi Denis-Courmont
>> 
>> contrib: use git-archive and save the hash
>> 
>> git-archive is supposed to validate the git repository to match the
>> git hash, and the git hash itself is supposed to be cryptographically
>> secure. So we if save it, we can reuse it to check the tarball.
>> 
>> Note: Alternatively, we could rely on git-archive making reproducible
>> tarballs, which it indeed does, then perform a regular hash of the
>> decompressed tarball. It is however unclear if git-archive tarballs
>are
>> reproducible *across* Git versions, which could lead to problems.
>> 
>> Note 2: This still requires a head to fetch from. In principles,
>> fetching a hash directly is possible, but not all servers allow it.
>> 
>> >
>http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=8fe4168947dd6cabd2aabb84cebc9aa6148e5d43
>> ---
>> 
>>  contrib/src/main.mak | 30 +++++++++++++++++++++++-------
>>  1 file changed, 23 insertions(+), 7 deletions(-)
>> 
>> diff --git a/contrib/src/main.mak b/contrib/src/main.mak
>> index 9461a6c..6d95f56 100644
>> --- a/contrib/src/main.mak
>> +++ b/contrib/src/main.mak
>> @@ -252,6 +252,12 @@ else
>>  XZCAT ?= $(error xz and lzma client not found!)
>>  endif
>>  
>> +ifeq ($(shell which xz >/dev/null 2>&1 || echo FAIL),)
>> +XZ = xz
>> +else
>> +XZ ?= $(error XZ (LZMA) compressor not found!)
>> +endif
>> +
>>  ifeq ($(shell which bzcat >/dev/null 2>&1 || echo FAIL),)
>>  BZCAT = bzcat
>>  else
>> @@ -317,13 +323,23 @@ HOSTVARS_PIC := $(HOSTTOOLS) \
>>  	LDFLAGS="$(LDFLAGS)"
>>  
>>  download_git = \
>> -       rm -Rf $(@:.tar.xz=) && \
>> -       $(GIT) clone $(subst HEAD, --depth 1,$(findstring HEAD, $3))
>> $(2:%=--branch %) $(1) $(@:.tar.xz=) && \
>> -       (cd $(@:.tar.xz=) && $(GIT) checkout $(3:%= %)) && \
>> -       rm -Rf $(@:%.tar.xz=%)/.git && \
>> -       (cd $(dir $@) && \
>> -       tar cvJ $(notdir $(@:.tar.xz=))) > $@ && \
>> -       rm -Rf $(@:.tar.xz=)
>> +       rm -Rf -- "$(@:.tar.xz=)" && \
>> +       $(GIT) init --bare "$(@:.tar.xz=)" && \
>> +       (cd "$(@:.tar.xz=)" && \
>> +       $(GIT) remote add origin "$(1)" && \
>> +       $(GIT) fetch origin "$(2)") && \
>> +       (cd "$(@:.tar.xz=)" && \
>> +       $(GIT) archive --prefix="$(notdir $(@:.tar.xz=))" \
>> +               --format=tar "$(3)") > "$(@:.xz=)" && \
>> +       echo "$(3) $(@)" > "$(@:.tar.xz=.githash)" && \
>> +       rm -Rf -- "$(@:.tar.xz)" && \
>> +       $(XZ) --stdout "$(@:.xz=)" > "$@.tmp" && \
>> +       mv -f -- "$@.tmp" "$@"
>> +check_githash = \
>> +       h=`sed -n -e "s,^\([0-9a-fA-F]\{40\}\) $<,\1,p" \
>> +               < "$(<:.tar.xz=.githash)"` && \
>> +       test "$$h" = "$1"
>> +
>>  checksum = \
>>  	$(foreach f,$(filter $(TARBALLS)/%,$^), \
>>  		grep -- " $(f:$(TARBALLS)/%=%)$$" \
>> 
>> _______________________________________________
>> vlc-commits mailing list
>> vlc-commits at videolan.org
>> https://mailman.videolan.org/listinfo/vlc-commits
>
>
>-- 
>Jean-Baptiste Kempf -  President
>+33 672 704 734
>_______________________________________________
>vlc-devel mailing list
>To unsubscribe or modify your subscription options:
>https://mailman.videolan.org/listinfo/vlc-devel

Yes. Wrong prefix is already fixed.

However, most if not all git-based rules have broken dependency/update propagation. This bug is preexistent: the name of the tarballs doed not depend on the checked-out version.
-- 
Rémi Denis-Courmont