Commit Graph

155 Commits

Author SHA1 Message Date
redfish 042db0bd0f cmake: cleanup logic that sets flags per target/subdir
The previous logic that used a COMMON_*_FLAGS intermediate variable
and then re-assigned CMAKE_*_FLAGS before including each subdirectory
was confusing and ugly. This PR is the right way to do it.

This commit is purely refactoring: built binaries unchanged.
2016-07-28 21:20:50 -04:00
redfish 0f990d0183 cmake,common: flag for stack trace
By default the flag is enabled whenever libunwind is found on the
system, with the exception of static build on OSX (for which we can't
install the throw hook #932 due to lack of support for --wrap in OSX
ld64 linker).
2016-07-27 01:52:33 -04:00
redfish 4b3a7885ac cmake: install throw hook in OSX build too
This is an attempt to fix build with STATIC=ON on OSX (#932):

[ 95%] Linking CXX executable ../../bin/bitmonerod Undefined symbols for
architecture x86_64: "___real___cxa_throw", referenced from:
___wrap___cxa_throw in libcommon.a(stack_trace.cpp.o) ld: symbol(s) not found
for architecture x86_64
2016-07-26 02:30:59 +00:00
redfish 3c92c2f096 cmake: do not install hook on throw when building tests
This fixes build of tests with STATIC=ON, which failed with:

/tmp/cc8lNtqY.ltrans12.ltrans.o: In function
`boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::thread_resource_error>
>::rethrow() const [clone .lto_priv.41]':
cc8lNtqY.ltrans12.o:(.text+0x4e): undefined reference to `__wrap___cxa_throw'

The hook is implemented in libcommon, which is not linked into some of the test
binaries.  An alternative solution is to link all tests against libcommon,
but that seems worse because it introduces a false dependency (also,
I tried that and for some of the test binaries the linker still failed to
pick up the symol from libcommon, strangely.)
2016-07-26 02:20:16 +00:00
redfish dca9fe175c cmake: do not pass -Werror when building tests
The tests currently issue a warning that
"warning: -fassociative-math disabled; other options take precedence"

The associative math optimization is turned on indirectly by -Ofast.
Apparently, the optimization is forced to be disabled, while compiling
test harnesses generated by Google Test framework.

Unfortunately, there is no -Wno-error=* flag to disable this warning
(see gcc --help=warnings).

An alternative to this patch is to disable the optimization explicitly
with -fno-associative-math, but that seems worse.

Another alternative is to not pass -Ofast for tests build, but we
want the tests to be built with exact same optimization flags as
the code being tested, otherwise the value of the tests is diminished.

Another alternative is to remove -Werror from the entire build, but
it's good to include that flag to preclude people leaving warnings.

A note regarding implementation of not passing -Werror for tests:
I considered filtering out -Werror from CMAKE_{C,CXX}_FLAGS but
that seems to be worse because it's surprizing behavior, to those
reading the code that adds -Werror. It is better to add it for
when it is used and not added otherwise. I also considered relying
on order, adding -Werror after inluding 'tests' subdir, but before
including the other subdirs, but that also seems cryptic to the
reader. So, I settled with the current solution, of explicitly
setting CMAKE_{C,CXX}_FLAGS to different values before including the
respective subdir.

Testing done: compared compiler invocation for non-tests source files
using `make VERBOSE=1` with and without this commit: the only difference
is the position of -Werror. So, this commit doesn't change the binary.
2016-07-22 18:08:33 -04:00
Riccardo Spagni 48e14ef66c
fix atomic library to only fire off for clang 2016-07-20 14:25:59 +02:00
Riccardo Spagni 39f7fad71d
fix duplicate clang block in cmake 2016-07-20 14:24:50 +02:00
Riccardo Spagni 77015e8132
Merge pull request #904
f07f120 cmake: don't try to link with atomic on Apple (redfish)
19349d7 cmake: ARM: clang: make warning non-fatal: inline asm (redfish)
f3e09f3 cmake: link with -latomic for clang (redfish)
f4b35ae cmake: include -ldl via cmake built-in var (redfish)
fa85cd8 common: stack trace: make clang happy with func ptrs (redfish)
4dce26b cmake: do not pass -stdlib=c++ to clang >=3.7 (redfish)
2016-07-20 13:56:59 +02:00
Riccardo Spagni 255ff79416
Merge pull request #896
1e89f4f cmake: do not ignore dangerous warnings with -Wno-error (redfish)
2016-07-20 13:51:54 +02:00
redfish f07f120e5d cmake: don't try to link with atomic on Apple
-latomic is necessary with Clang on Linux, but
apparently, on OSX it's not found.
2016-07-12 00:25:56 -04:00
redfish 19349d7870 cmake: ARM: clang: make warning non-fatal: inline asm
Clang issues a warning for some inline asm in stack_trace.cpp.  This
patch ieaves the warning to be displayed as a reminder to fix
the code.
2016-07-10 21:27:00 -04:00
redfish f3e09f36d3 cmake: link with -latomic for clang
otherwise clang build fails with

../cryptonote_core/libcryptonote_core.a(miner.cpp.o): In function
`std::__atomic_base<unsigned long long>::load(std::memory_order) const':
/usr/bin/../lib/gcc/i686-pc-linux-gnu/6.1.1/../../../../include/c++/6.1.1/bits/atomic_base.h:396:
undefined reference to `__atomic_load_8'

This has no effect on the gcc build.

The one strange thing is that test code like

std::atomic<int> x;
int main() { return x; }

compiles and links without errors with clang, without -latomic.  This
alone would suggest that this patch is unnecessary, but that is not the
case. It's not clear exactly why, though. The bitmonero code is
including the same header, but it must be doing something more complex
than in this test code snippet that causes the failure at link time
pasted above. In any case, passing -latomic fixes the problem and
seems safe.

.
2016-07-10 21:27:00 -04:00
redfish f4b35aeafd cmake: include -ldl via cmake built-in var
This does two things:
1. fixes clang build, which otherwise errors with undefined symbol
'dlsym'.
2. simplifies the cmake script, delegating to cmake to figure
   out platform-specific flags for linking against the dl library.
2016-07-10 21:27:00 -04:00
redfish 4dce26bba4 cmake: do not pass -stdlib=c++ to clang >=3.7
Tested on Linux (Arch) with clang 3.7 and 3.8 i686 and ARM:
if -stdlib=c++ is passed to clang, then the build errors
out with <string>,<iostrea>,etc. headers not found. Simply
not passing the arg fixes the problem.

**NOTE**: not tested on OSX.
2016-07-10 21:19:16 -04:00
redfish 1e89f4f2c5 cmake: do not ignore dangerous warnings with -Wno-error
Shorten the list of warnings that are reported, but
which are forced to NOT generate an error, via -Wno-error.

Unwhitelist these: strict-aliasing, sign-compare, type-limits

For example, ignoring strict-aliasing warning caused
lots of wasted time diagnosing Issue #847.
2016-07-09 11:59:07 -04:00
redfish ab250754f2 cmake: pass -fno-strict-aliasing to fix build with GCC 6.1.1
This patch is a temprorary workaround until aliasing is fixed in the
code (Issue #847).
2016-07-08 21:42:05 -04:00
Riccardo Spagni 65df33fb06
Merge pull request #877
ecd0f2d makefile: remove unnecessary ARM-specific targets (redfish)
c650160 cmake: disable AES on ARM (redfish)
804b1bc cmake: initialize ARCH for native builds (redfish)
bb39034 cmake: remove repetitive code in handling of NO_AES (redfish)
2016-07-06 18:14:03 +02:00
Riccardo Spagni 628f57b212
Merge pull request #873
d718960 remove POSIX_C_SOURCE and remove dlfcn.h for static builds (luigi1111)
2016-07-06 18:12:03 +02:00
Ilya Kitaev 653c7e3804 cmake: BUILD_TESTS as option explicitly; added missed dependency 2016-06-23 16:01:33 +03:00
redfish c6501607f3 cmake: disable AES on ARM
On ARM, gcc fails with 'unknown argument' if passed '-maes'.
2016-06-21 22:32:55 -04:00
redfish 804b1bc197 cmake: initialize ARCH for native builds
We need ARCH, because it needs to be set for ARM7, ARM6 to be
initialized.

Strangely, on different machines (both ARMv7, Arch), ${ARCH}
var is either empty or 'native'. Handle both cases.
2016-06-21 22:32:16 -04:00
redfish bb39034405 cmake: remove repetitive code in handling of NO_AES
Minor eefactor only. No behavior or interface change in this commit.
2016-06-21 19:53:07 -04:00
luigi1111 d7189600ae
remove POSIX_C_SOURCE and remove dlfcn.h for static builds
The former was a faulty "fix" for gmtime_r not existing on Windows. The latter is needed only for dynamic builds, and is not included with msys2, which ends up fine because Windows is only built static at this time.
2016-06-21 11:16:25 -05:00
Riccardo Spagni fd32a3b52a
Merge pull request #869
03e545d More fixing for Windows (iDunk)
e8baa57 CMakeLists.txt: fix build for mingw (iDunk)
2016-06-21 09:32:12 +02:00
iDunk 03e545d323
More fixing for Windows 2016-06-20 22:38:36 +01:00
iDunk e8baa57326
CMakeLists.txt: fix build for mingw
Fix from iDunk from IRC.
2016-06-20 21:39:13 +01:00
moneromooo-monero f72388c1c6
CMakeLists: fix build without libunwind 2016-06-20 19:20:14 +01:00
moneromooo-monero e409e59d29
Print stack trace on exceptions
if libunwind is found.

Useful for debugging logs.
2016-04-28 23:34:51 +01:00
moneromooo-monero f7301c3563
Revert "Print stack trace upon exceptions"
Ain't nobody got time for link/cmake skullduggery.

This reverts commit fff238ec94.
2016-03-21 10:12:23 +00:00
moneromooo-monero fff238ec94
Print stack trace upon exceptions
Useful for debugging users' logs
2016-03-19 21:48:36 +00:00
Riccardo Spagni 11de19266b
change to LMDB as default, even on 32-bit and ARM 2016-03-12 21:20:00 +02:00
warptangent 9832d18dca
cmake: Include OpenSSL libraries in static linking 2016-02-12 16:54:14 -08:00
warptangent c2f2437586
cmake: Remove unused variable
DL is empty and unused elsewhere.

The intention at one point may have been to use CMAKE_DL_LIBS, but that
would more likely apply in some situations involving static linking.
2016-02-12 15:02:12 -08:00
warptangent 7205210b0f
cmake: Fix unbound config compile settings
This allows the OpenSSL function checks to compile in unbound's CMake
configuration.

Otherwise, the functions SHA256() and EVP_sha512() won't be called from
libunbound as possible algorithms.

They had not been compiling because static OpenSSL libraries were being
used, along with lack of -ldl. The static library preference is
unnecessary for the checks, so use default suffixes ordering for
CMAKE_FIND_LIBRARY_SUFFIXES when building unbound.

Related files:
configure_checks.cmake
external/unbound/validator/val_secalgo.c
  secalgo_ds_digest(), setup_key_digest()
2016-02-12 15:02:06 -08:00
warptangent 3d1e690cb2
Add make option to disable BerkeleyDB
Sample use:
BERKELEY_DB=0 make debug

This makes development with BlockchainLMDB easier when virtual methods
have changed and don't match BlockchainBDB.
2016-02-01 13:06:15 -08:00
me0wmix 34ebfe39cd Last fixups in the Cmake stuff and better default build instruction 2016-01-22 20:27:01 -07:00
me0wmix c965bf4a7c Added/corrected OpenBSD build instructions. 2016-01-21 11:34:02 -07:00
me0wmix 28f95eb001 OpenBSD support for Monero. 2016-01-21 11:18:26 -07:00
Howard Chu 94de39fbad Tweak arm6 flags 2016-01-11 23:14:06 +00:00
Howard Chu d26e7d0b3a ARMv7 optimization flags are too aggressive 2016-01-03 12:58:08 +00:00
Howard Chu 26aac68095 Fix release flags
flags were being accumulated instead of being set.
Win32 was getting -O2 -DNDEBUG -Ofast -DNDEBUG etc...
Also for Win32, stay compatible to WinXP
2016-01-02 21:53:02 +00:00
Riccardo Spagni de03926850
updated copyright year 2015-12-31 08:39:56 +02:00
Howard Chu cfe15b81bd Also set stacksize for Win32 not on MSVC 2015-12-20 18:41:36 +00:00
warptangent 725acc7f17
Replace tabs with two spaces for consistency with rest of codebase
Remove trailing whitespace in same files.
2015-12-15 06:22:06 -08:00
moneromooo-monero 180bcde866
build: default to Berkeley DB for 32 bit and ARM 2015-10-04 19:01:56 +01:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo 464dbe7379 Fixed MINGW duplicate library errors when MSYS folder cannot be found.
*Thanks to luigi1111 for reporting.
2015-07-17 19:49:22 -07:00
warptangent bbb936df0f
Use g++ flag "-pthread" instead of "-lpthread", where preferred
CMake supports this through THREADS_PREFER_PTHREAD_FLAG.

Remove inclusion of pthread library in EXTRA_LIBRARIES, as the
individual CMakeLists.txt files which need pthread already require it
with CMAKE_THREAD_LIBS_INIT.
2015-07-16 05:30:46 -07:00
Riccardo Spagni 54c785d0bb
fixed ARM arch test to prevent breaking on short arch strings 2015-07-16 12:46:38 +02:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
moneromooo-monero 8623492150
Interpret x86_64 as x86-64 for architecture 2015-07-09 14:56:20 +01:00