[Android] Fwd: [UPX] android shared objects without UPX

[Rafaël Carré]

Hi guys, still working on compressing the .so :

12277208 libvlcjni.so
6336200  libvlcjni.so.gz	# gzip -9
6277310  libvlcjni.so.bz2	# bzip2
5034493  libvlcjni.so.7z	# 7z
5026124  libvlcjni.so.xz	# xz
4954633  libvlcjni.so.paq8	# zpaq : 43 seconds to decompress on Core2Duo
2.26Ghz

I didn't test yet but I expect the size of the decompressor to be well
below 100K.

http://www.7-zip.org/sdk.html has public domain 7z decompressor in Java,
C, asm etc...

We could either use /data/data/something as temporary storage and delete
the decompressed .so on exit, or leave it there permanently (and just
decrease the size of download without effect on internal storage).

I expect /tmp to be noexec like /sdcard but I didn't try yet.
Using RAM storage would be perfect to decompress libvlcjni.so

-------- Message original --------
Date: Thu, 19 Jul 2012 15:24:21 -0700
From: John Reiser <jreiser at bitwagon.com>

Hi Rafaël,

So, taking a more global view than just how to involve UPX:
You should consider this strategy, too, for constructing
compressed shared objects for android (or any other environment
which has dlopen):

1. Build the usual.so in the normal way.

2. Build an impostor .so using some compression of the whole
   usual.so file, then arrange supply the uncompressed usual.so
   by "whole file" decompression and dlopen() at runtime.

   I will use gzip/zlib fpr an example because it has a
   subroutine API zlib.h.  Other compression will work as long
   as there is a subroutine API to compress and uncompress arrays
   (instead of files.)

   Compress the input file usual.so, which generates the
   output file usual.so.gz:  gzip usual.so
   Compile the usual.so.gz file into a data array:
   ----- cpr_usual.S
   cpr_usual_begin: .global cpr_usual_begin
	.incbin "usual.so.gz"
   cpr_usual_end: .globl cpr_usual_end
   -----
   Build another shared library "impostor.so" with an
   empty main() function: "main() {}"
   and an initializer function (DT_INIT function)
   which uses the zlib subroutine API
   to decompress the cpr_usual_begin[] array, and write the
   de-compressed usual.so into a file such as /tmp/usual.so.
   Then the initializer function calls dlopen("/tmp/usual.so", ...);
   (A more refined initializer function can check for /tmp/usual.so
   existing already, etc.)

3. Link client programs with usual.so first, to be sure that
   there are no undefined external symbols.

4. In the runtime environment, replace /usr/lib/usual.so
   with impostor.so .

In most cases the client won't care about the "middle man"
impostor.so, as long as the symbols get resolved by the
dlopen()ed /tmp/usual.so.  This ignores runtime name conflicts
(two different definitions of the same symbol name, with the
result depending on the search order).  This also does not work
if the client cares too much about which library satisfies the
request for an external symbol. (There are some tricks to play
with DT_SONAME, but an overly-paranoid client might be unhappy.)

There is some messiness with the file /tmp/usual.so.
There is some messiness with multiple simultaneous clients.
But this gives you a different approach which you can try now,
instead of waiting for UPX.  You can also try compressors that
UPX does not offer, such as xz (although "upx --lzma" ought to be
very close to xz).

-- 
John Reiser