Minimal implementation to strip the [largest files][lf-gist] by
default, mostly made up of gcc/binutils and test programs.
gdal and geos both produce large libraries, but the libs themselves
aren't worth stripping, it's the 20 odd programs produced by gdal
with those libs statically linked that consume the most space.
I'm leaving these undocumented as the defaults seems reasonable and
the interface may well change when we enable debug/release variants.
closes#985closes#1249
[lf-gist]:https://github.com/mxe/mxe/issues/1249#issuecomment-193392038
The following script was applied:
sed ':a;/part of MXE.$/{N;s/\n//;ba}' -i $(git grep -l 'part of MXE')
sed 's/\(part of MXE\).*\(See index.html\)/\1. \2/' -i \
$(git grep -l 'part of MXE.*See index.html')
before='This file is part of MXE. See index.html for further information.'
after='This file is part of MXE. See LICENSE.md for licensing information.'
sed "s/$before/$after/" -i $(git grep -l 'part of MXE')
Then git grep 'index.html for further information' revealed two other files.
One of them was patched manually (patch.mk). Makefile has text
"See index.html for further information" unrelated to licensing.
See https://github.com/mxe/mxe/issues/1500#issuecomment-241340792
see http://lists.nongnu.org/archive/html/mingw-cross-env-list/2014-07/msg00002.html
Many FTP servers block connections from Tor and some
VPN servers. HTTP servers don't do this normally.
Example of failed FTP download attempt of binutils-2.24.tar.bz:
$ torsocks wget ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.24.tar.bz2
--2014-07-20 13:26:48-- ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.24.tar.bz2
=> `binutils-2.24.tar.bz2'
Resolving ftp.gnu.org (ftp.gnu.org)... 208.118.235.20
Connecting to ftp.gnu.org (ftp.gnu.org)|208.118.235.20|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /pub/gnu/binutils ... done.
==> SIZE binutils-2.24.tar.bz2 ... 22716802
==> PASV ... done. ==> RETR binutils-2.24.tar.bz2 ...
Error in server response, closing control connection.
Retrying.
Same package was downloaded via HTTP successfully:
$ torsocks wget http://ftp.gnu.org/pub/gnu/binutils/binutils-2.24.tar.bz2
--2014-07-20 13:32:37-- http://ftp.gnu.org/pub/gnu/binutils/binutils-2.24.tar.bz2
Resolving ftp.gnu.org (ftp.gnu.org)... 208.118.235.20
Connecting to ftp.gnu.org (ftp.gnu.org)|208.118.235.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22716802 (22M) [application/x-bzip2]
Saving to: `binutils-2.24.tar.bz2'
100%[=================>] 22,716,802 721K/s in 24s
2014-07-20 13:33:03 (915 KB/s) - `binutils-2.24.tar.bz2' saved [22716802/22716802]
Trying download from Tor Browser, I get error message:
425 Security: Bad IP connecting.
HTTP URLs were added to FTP URLs-only packages.
In many cases, ftp://ftp.gnu.org can be accessed
from http://ftp.gnu.org
If both URLs of a package are FTP, then one of them
was replaced with HTTP.
Command to check that those packages can be build successfully
if behind Tor:
$ torsocks make autoconf automake binutils bison cloog coreutils file freetds gcc gdb gettext gmp gnutls gperf isl libbluray libffi libgcrypt libgpg_error libidn libmicrohttpd libpng libxml2 libxslt m4 pthreads-w32 sed dcmtk mpfr
I've run the test above successfully.
Since we need pkgconf for Cflags.private support, it seems like a
logical place for these types of config files instead of being
spread over Makefile, binutils, and gcc.
Most of the automatic conversion was done using the following Python script:
import os
import re
pkgs = sorted(mkfile[:-len('.mk')] for mkfile in os.listdir('src') if mkfile.endswith('.mk'))
with open('index.html', 'rb') as f:
index = f.read()
pkgs_index = re.findall('<td class="package">([^<]*)</td>', index)
assert pkgs_index == pkgs
versions = dict(re.findall('<td id="([^"]*)-version">([^<]*)</td>', index))
assert sorted(versions.keys()) == pkgs
for pkg in pkgs:
version = versions[pkg]
with open('src/' + pkg + '.mk', 'rb') as f:
mk = f.read()
checksumpos = mk.index('\n$(PKG)_CHECKSUM ')
versionline = '\n$(PKG)_VERSION := %(version)s' % {'version': version}
newmk = mk[:checksumpos] + versionline + mk[checksumpos:]
with open('src/' + pkg + '.mk', 'wb') as f:
f.write(newmk)
Tony Theodore
Viktor Szakats
Boris Nagaev
Mark Brand
John W. Eaton
Niels Kristian Bech Jensen
Volker Grabsch
Volker Grabsch