[x264-devel] [PATCH] build: Replace cltostr.pl by a shell script

[Steve Borho]

On Wed, May 7, 2014 at 10:04 AM, Diego Biurrun <diego at biurrun.de> wrote:
> This avoids a dependency on Perl to build OpenCL support.
> ---
>  Makefile         |  2 +-
>  configure        |  9 --------
>  tools/cltostr.pl | 65 --------------------------------------------------------
>  tools/cltostr.sh | 30 ++++++++++++++++++++++++++
>  4 files changed, 31 insertions(+), 75 deletions(-)
>  delete mode 100644 tools/cltostr.pl
>  create mode 100755 tools/cltostr.sh

I believe I implemented this the first time with od and someone
complained it didn't exist in their MSYS tree, and so I tried it
several other ways and finally ended up with the perl script that was
eventually checked in.

In retrospect, od should be more likely to be present than perl so
this patch is fine with me.

Answering questions in Janne's email:

The only real requirement here is that the generated C array can
compile to the same OpenCL binary as the original source files, so
white-space in oclobj.h is trivial. Pruning white-space and comments
from the OCL code just makes the compiled binary a bit smaller.

The md5 hash of the OCL source is copied into the compiled binary
cache file, as a way to later detect if the compiled binary is
out-of-date.  So if the hash generation failed, that dependency
detection would be broken.

> diff --git a/Makefile b/Makefile
> index 4cdb2e7..26c7c25 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -125,7 +125,7 @@ endif
>
>  ifeq ($(HAVE_OPENCL),yes)
>  common/oclobj.h: common/opencl/x264-cl.h $(wildcard $(SRCPATH)/common/opencl/*.cl)
> -       cat $^ | perl $(SRCPATH)/tools/cltostr.pl x264_opencl_source > $@
> +       cat $^ | $(SRCPATH)/tools/cltostr.sh $@
>  GENERATED += common/oclobj.h
>  SRCS += common/opencl.c encoder/slicetype-cl.c
>  endif
> diff --git a/configure b/configure
> index 67cdf05..697ae17 100755
> --- a/configure
> +++ b/configure
> @@ -1050,15 +1050,6 @@ ASFLAGS="$ASFLAGS -DBIT_DEPTH=$bit_depth"
>  libdl=""
>  if [ "$opencl" = "yes" ]; then
>      opencl="no"
> -    log_check "for perl"
> -    output=$(perl -v)
> -    if [ "$output" = "" ]; then
> -        log_fail
> -        echo 'OpenCL support requires perl to compile.'
> -        echo 'use --disable-opencl to compile without OpenCL.'
> -        exit 1
> -    fi
> -    log_ok
>      # cygwin can use opencl if it can use LoadLibrary
>      if [ $SYS = WINDOWS ] || ([ $SYS = CYGWIN ] && cc_check windows.h "" "LoadLibraryW(0);") ; then
>          opencl="yes"
> diff --git a/tools/cltostr.pl b/tools/cltostr.pl
> deleted file mode 100644
> index a90e6a9..0000000
> --- a/tools/cltostr.pl
> +++ /dev/null
> @@ -1,65 +0,0 @@
> -# Perl script used for compiling OpenCL src into x264 binary
> -#
> -# Copyright (C) 2013-2014 x264 project
> -# Authors: Steve Borho <sborho at multicorewareinc.com>
> -
> -use Digest::MD5 qw(md5_hex);
> -
> -# xxd takes a VAR, which will be the variable name
> -# and BYTES, a string of bytes to beencoded.
> -sub xxd
> -{
> -  my %args = @_;
> -  my $var = $args{VAR};
> -  my $bytes = $args{BYTES};
> -  my @hexbytes;
> -  my @bytes = split //, $$bytes;
> -  foreach $b (@bytes)
> -  {
> -    push @hexbytes, sprintf("0x%02X", ord($b));
> -  }
> -
> -  # Format 'em nice and pretty-like.
> -  print 'static const char ' . $var . '[] = {' . "\n";
> -  my $count = 0;
> -  foreach my $h (@hexbytes)
> -  {
> -    print "$h, ";
> -    $count++;
> -    if ($count == 16)
> -    {
> -      print "\n";
> -      $count = 0;
> -    }
> -  }
> -  print "\n0x00 };\n\n";
> -
> -  return;
> -}
> -
> -if (@ARGV < 1)
> -{
> -  printf "%s: VARNAME ", $0 . "\n";
> -  exit(-1);
> -}
> -
> -
> -my @lines;
> -while(<STDIN>)
> -{
> -  s/^\s+//;                # trim leading whitespace
> -  if (/^\/\//)
> -  {
> -    next;   # skip the line if it starts with '//'
> -  }
> -  push @lines, $_;
> -}
> -
> -my $lines = join '', @lines;
> -xxd(VAR => @ARGV[0], BYTES => \$lines);
> -
> -my $hash = md5_hex($lines);
> - at hash = ( $hash =~ m/../g );
> -
> -
> -xxd(VAR => @ARGV[0] . "_hash", BYTES => \$hash);
> diff --git a/tools/cltostr.sh b/tools/cltostr.sh
> new file mode 100755
> index 0000000..c8b544e
> --- /dev/null
> +++ b/tools/cltostr.sh
> @@ -0,0 +1,30 @@
> +#!/bin/sh
> +# Convert standard input to a C char array, write to a file, then create an
> +# MD5 sum of that file and append said MD5 sum as char array to the file.
> +
> +FILE=$1
> +
> +# Filter out whitespace, empty lines, and comments.
> +sanitize() {
> +    sed 's/^[[:space:]]*//; /^$/d; /^\/\//d'
> +}
> +
> +# Convert stdin to a \0-terminated char array.
> +dump() {
> +    printf 'static const char %s[] = {\n' $1
> +    od -v -A n -t x1 | sed 's/.\(..\)/0x\1, /g'
> +    printf '0x00 };\n'
> +}
> +
> +# Print MD5 hash w/o newline character to not embed the character in the array.
> +hash() {
> +    # md5sum is not standard, so try different platform-specific alternatives.
> +    { md5sum $1 2> /dev/null || md5 -q $1 || digest -a md5 $1; } |
> +        cut -b -32 | tr -d '\n\r'
> +}
> +
> +sanitize |
> +    dump x264_opencl_source > $FILE
> +
> +hash $FILE |
> +    dump x264_opencl_source_hash >> $FILE
> --
> 1.8.3.2

--
Steve Borho