Commit Graph

462 Commits

Author SHA1 Message Date
Tony Theodore c0c8d47d6a Makefile: add support for cross-target and order-only deps 2017-12-23 08:18:52 +11:00
Tony Theodore 0e1b95fc24 Improve handling of virtual packages and build messages
- add metadata to distinguish virtual package types
  - improve build order and status messages for empty packages
        print pkg type or `disabled` instead of `no-build`
  - add BUILD_DRY_RUN option and fixup make -t
  - update build-matrix rule to identify virtual pkgs
2017-12-23 08:18:52 +11:00
Tony Theodore 51217e01d2 updates: move to separate file and add generic updater function 2017-11-18 16:19:55 +11:00
Tony Theodore a3624b48b1 Makefile: speedup LOOKUP_PKG_RULE
LOOKUP_PKG_RULE is called many times and spends a lot of time searching
for rules when the default rule is the most common. This simply
avoids exhaustive searching since we can determine beforehand if there
are multiple rules.

Also drops some of the gmsl functions if favour of normal make
variables.

First commit adds sort/strip so print-deps-for-build-pkg output can be
easily compared.

```
$ git checkout HEAD~1
$ time make print-deps-for-build-pkg \
      MXE_TARGETS="`echo {i686-w64-mingw32,x86_64-w64-mingw32}.{static,shared}`"> orig
real	0m17.564s
user	0m16.410s
sys	0m0.783s

$ git checkout speedup
$ time make print-deps-for-build-pkg \
      MXE_TARGETS="`echo {i686-w64-mingw32,x86_64-w64-mingw32}.{static,shared}`"> new
real	0m4.036s
user	0m3.624s
sys	0m0.326s

$ diff -u orig new
$
```
2017-11-13 20:51:08 +11:00
Tony Theodore 9f8fb60f8a print-deps-for-build-pkg: sort and strip output for repeatable comparison 2017-11-13 19:56:53 +11:00
Tony Theodore 75417437a2 Makefile and s3: add mxe mirror and tidy output 2017-11-13 11:20:54 +11:00
Tony Theodore b82080ec4d Makefile: switch GitLab to https and reorder backup servers
See: https://github.com/mxe/mxe/pull/1899#issuecomment-343731128
  - GitLab redirects to https anyway
  - final fallthrough to Cloudfront will be http
  - GitLab is (currently) the most complete so try it first
2017-11-13 07:59:36 +11:00
Tony Theodore ba358185fa Makefile: also create suffixed archives for backup downloads
See:
https://github.com/mxe/mxe/pull/1681#issuecomment-280887497
2017-11-12 15:07:21 +11:00
Tony Theodore 1e9aa771e3 Makefile: try next download url on failed checksum
There are various cases where a download succeeds but the checksum
is wrong:
  - upstream re-releases tarball without new version
  - sourceforge returns 200 with a text message about outage
  - other strange glitches

In these cases, we currently fail and stop the build. This PR allows
for falling through the next URLs until it will eventually get to
the checksum-suffixed version.

This ensures builds are always (potentially) repeatable as the checksum
associated with the version is reachable. Next step would be to change
all the backups to use suffixed tarballs.
2017-11-12 13:56:36 +11:00
Tony Theodore c974863b7c Makefile: add more disable doc options
fixes #1936
2017-10-19 12:54:01 +11:00
Tony Theodore b58b3e9d3e Makefile: strip dot from $PATH
fixes #1893
2017-10-13 16:38:10 +11:00
Tony Theodore 3e6d76dec6 Makefile: always eval $(PKG)_PATCHES as a variable
Allows plugin packages to re-use the variable (eg. gcc7 and gcc-host)
and keep this metadata in sync (same as *_VERSION, *_FILE, etc.)
2017-10-09 15:19:53 +11:00
Viktor Szakats d735f7fe56 bump primary package mirror to HTTPS 2017-09-24 16:59:40 +00:00
Viktor Szakats b8599ed14b secure/update URLs
* change libssh2/apache/xiph URLs to canonical ones
* resubmit patch to remove unnecessary
  --no-check-certificate option. The URLs
  are non-HTTPS anyway
* blas: restore URL and point to versioned archive
2017-09-24 16:04:56 +00:00
Tony Theodore bec723f574 Makefile: add more MXE_DISABLE_DOC_OPTS 2017-07-20 20:21:31 +10:00
Boris Nagaev fcdb7c18d9 Merge pull request #1743 from tonytheodore/build-matrix
build-matrix: fix detection of native virtual packages
2017-05-20 21:32:17 +00:00
Boris Nagaev eba574d606 MXE_DISABLE_DOCS: add nodist_man* version
Tor installs its mans in nodist_man1_MANS.
2017-05-19 01:59:05 +02:00
Boris Nagaev 1aee39402f add dist_bin_SCRIPTS to MXE_DISABLE_PROGRAMS 2017-05-19 01:26:42 +02:00
Boris Nagaev ae26487a84 factor out var WGET_TOOL from WGET
It is needed to specify a path to custom wget binary without
removing the argument passed by WGET (--user-agent).

Also can be used to add --no-check-certificate back.
2017-05-19 00:42:48 +02:00
Tony Theodore 3bc28c1513 build-matrix: fix detection of native virtual packages 2017-04-03 22:34:23 +10:00
Tony Theodore d0aa402fe6 Makefile: explicitly set .DEFAULT_GOAL for LOCAL_PKG_LIST
Use `.DEFAULT_GOAL` [special variable] instead of `.DEFAULT`
[built-in target] as the latter depends on include order and
can cause confusing behaviour.

`LOCAL_PKG_LIST` was [broken] in #1729 and this allows for any
current `settings.mk` to work correctly. Should also fix #1264
and #1267 since MXE isn't going to include any other files that
set this variable in a surprising way.

[special variable]:https://www.gnu.org/software/make/manual/make.html#Special-Variables
[built-in target]:https://www.gnu.org/software/make/manual/make.html#Special-Targets
[broken]:1cf2afb5d3
2017-04-02 12:08:04 +10:00
Tony Theodore 1cf2afb5d3 Makefile: explicitly set .DEFAULT_GOAL
first target read can be affected by definitions in included files
2017-03-21 23:41:38 +11:00
Tony Theodore 6b40c10316 github: add check-gh-conf targets to verify updates and downloads 2017-03-21 15:59:32 +11:00
Boris Nagaev 13b5f5e047 add MANLINKS to MXE_DISABLE_DOCS
It is used in libotr (toolkit/Makefile.am) to define a list
of symlinks to the main man page. Search for MANLINKS reveals
that it is used in many other projects.
2017-03-20 17:35:18 +03:00
Tony Theodore b2d99407c0 Makefile: suppress messages in update-package 2017-03-19 12:51:54 +11:00
Tony Theodore 4ed90733c9 github: use `if` instead of `and` 2017-03-19 11:29:05 +11:00
Tony Theodore b4161379ee Makefile: add check-update-package-% target 2017-03-18 22:16:07 +11:00
Tony Theodore 96dd1ec4a0 add functions to standardise GitHub downloads and updates 2017-03-18 22:16:07 +11:00
Boris Nagaev 28e1101e65 Merge pull request #1660 from tonytheodore/cmake-conf
cmake: update and improve configuration with templates
2017-03-14 01:48:58 -07:00
Boris Nagaev a3ac9460ce use http version of gitlab.com backup
The rationale: bd2c77f4c2 (commitcomment-21229420)

> If you absolutely want to disable secure transport I think it'd be a
> better choice to use the http:// protocol instead, making the
> insecurity unambiguously visible in logs/screen output. (Not sure if
> GitLab supports it, but the first two does I think.)
2017-03-12 14:07:10 +01:00
Boris Nagaev 096c509099 no --no-check-certificate while updating packages
Regular downloads of packages are verified by checksums, so
--no-check-certificate doesn't compromise the build system,
but the checksums themselves are often updated with update-checksum-%
which in turn calls the regular package download mechanism, so there
is a possibility of downloading and sealing a poisoned file.

On the one hand, old systems may still rely on --no-check-certificate,
so it is not nice to completely disable it for regular downloads.
However keeping this option enabled for backup servers only is enough
to support such systems because of the fallback mechanism.
On the other hand, download from a backup doesn't make sense while
updating a package, because the package is definetely not in the backup yet.

So --no-check-certificate is now enabled only for backup servers
and backup servers are disabled while updating packages.

See https://github.com/mxe/mxe/pull/1694#issuecomment-285324739
2017-03-12 14:07:10 +01:00
Tony Theodore 24d37efa5d cmake: improve configuration with templates
- move cmake configuration from mxe-conf to cmake-conf
 - replace `echo` with templates for readability and maintenance
 - allow packages to set other dep files
 - set CMAKE_POLICY_DEFAULT_CMPNNNN in wrapper since
   `cmake_minimum_required` or `cmake_policy` can't be set in
   toolchain (closes #971)
2017-03-12 17:30:18 +11:00
Viktor Szakats 3166d4b715 add explicit protocol to backup urls 2017-03-10 22:17:46 +00:00
Viktor Szakats 45d34aaac7 more URL updates 2017-03-10 13:59:00 +00:00
Tony Theodore b8850b2bd0 Makefile: add [POSIX | WIN32]_THREADS variable 2017-02-07 14:52:48 +11:00
Tony Theodore 5da8168263 allow non-default exception handling targets 2017-02-07 14:33:26 +11:00
Tony Theodore 9bb609a73e Merge branch 'LuaAndC-levmar-ua' of https://github.com/tonytheodore/mxe into ua 2017-02-04 18:47:19 +11:00
Tony Theodore 48e792b9e7 Makefile: simplify setting default user agent 2017-01-21 16:49:18 +11:00
Tony Theodore 4fa1f12db8 Makefile: fix target validation checks
fixes #1497
2017-01-19 20:36:02 +11:00
Boris Nagaev ff36def5d3 add new packages backup: gitlab
See http://lists.nongnu.org/archive/html/mingw-cross-env-list/2017-01/msg00005.html
2017-01-08 03:17:31 +01:00
Boris Nagaev 2a9736d370 make core to be the first item of MXE_PLUGIN_DIRS
see https://github.com/mxe/mxe/issues/1614
2016-12-26 02:29:44 +01:00
Boris Nagaev 23c5e209d4 add descriptions and websites to packages.json
See https://github.com/mxe/mxe/issues/1422
2016-12-22 22:28:29 +01:00
Boris Nagaev 5d832ebf70 core packages are now a plugin
See https://github.com/mxe/mxe/issues/1422
2016-12-22 22:25:48 +01:00
Boris Nagaev f06ef1c01b mv docs/versions.json docs/packages.json
See https://github.com/mxe/mxe/issues/1422
2016-12-22 22:25:48 +01:00
Boris Nagaev f1a5deded9 fix detection of openssl in Debian Sid
$ openssl --help
Invalid command '--help'; type "help" for a list.
$ echo $?
1

$ openssl help
...
$ echo $?
0

fix https://github.com/mxe/mxe/issues/1593
2016-12-14 00:44:51 +01:00
Boris Nagaev f82e39dcef cleanup-deps-style: omit pkgs with derived deps
Do not touch lines like this:

  $(PKG)_DEPS     := $(patsubst $(TOP_DIR)/src/%.mk,%,\
  ...

(src/qt5.mk)
2016-12-05 09:40:54 +01:00
Boris Nagaev 1e9d757021 make cleanup-deps-style aware of plugins 2016-12-05 09:40:53 +01:00
Boris Nagaev 592c429875 fix updating docs/versions.json with plugins
fix https://github.com/mxe/mxe/issues/1580
2016-12-05 09:40:53 +01:00
Boris Nagaev 894608bf4d pkg download: do not append to previous attempt
Slow HTTP server with lag for tests:
https://gist.github.com/45472174f8bd97989c1eb9bc64d50468

Fix https://github.com/mxe/mxe/issues/1552
2016-12-05 09:39:36 +01:00
Boris Nagaev 634054b51d add MXE_PLUGIN_DIRS to autogenerated settings.mk 2016-11-19 01:15:43 +03:00