diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a3c59a..79891eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,166 +1,167 @@ # Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## Unreleased ### Changed - `CXXFLAGS` are now passed to the linker - Added this changelog - Using absolute paths for environmental variables when running `scons test` - Reorganized documentation layout - Gave the build system a facelift (docs are now generated directly with SCons instead of a Makefile) ### Deprecated - Python 2 support is discontinued. Version `v2.2.1` is the last PyPi build with a Python 2 wheel. ## Fixed - `UVWDumper` no longer imports `mpi4py` in sequential +- Compiling with different Thrust/FFTW backends ## v2.2.1 -- 2021-03-02 ### Added - Output registered fields and dumpers in `print(model)` - Added `operator[]` to the C++ model class (for fields) - Added `traction` and `displacement` properties to Python model bindings - Added `operators` property to Python model bindings, which provides a dict-like access to registered operators - Added `shape` and `spectrum` to properties to Python surface generator bindings - Surface generator constructor accepts surface global shape as argument - Choice of FFTW thread model ### Changed - Tests use `/tmp` for temporary files - Updated dependency versions (Thrust, Pybind11) ### Deprecated - Most `get___()` and `set___()` in Python bindings have been deprecated. They will generate a `DeprecationWarning`. ### Removed - All legacy code ## v2.2.0 -- 2020-12-31 ### Added - More accurate function for computation of contact area - Function to compute deviatoric of tensor fields - MPI implementation - Convenience `hdf5toVTK` function - Readonly properties `shape`, `global_shape`, `boundary_shape` on model to give shape information ### Changed - Preprocessor defined macros are prefixed with `TAMAAS_` - Moved `tamaas.to_voigt` to `tamaas.compute.to_voigt` ### Fixed - Warning about deprecated constructors with recent GCC versions - Wrong computation of grid strides - Wrong computation of grid sizes in views ## v2.1.4 -- 2020-08-07 ### Added - Possibility to generate a static `libTamaas` - C++ implementation of DFSANE solver - Allowing compilation without OpenMP ### Changed - NetCDF dumper writes frames to a single file ### Fixed - Compatibility with SCons+Python 3 ## v2.1.3 -- 2020-07-27 ### Added - Version number to `TamaasInfo` ### Changed - Prepending root directory when generating archive ## v2.1.2 -- 2020-07-24 This release changes some core internals related to discrete Fourier transforms for future MPI support. ### Added - Caching `CXXFLAGS` in SCons build - SCons shortcut to create code archive - Test of the elastic-plastic contact solver - Paraview data dumper (`.pvd` files) - Compression for UVW dumper - `__contains__` and `__iter__` Python bindings of model - Warning message of possible overflow in Kelvin ### Changed - Simplified `tamaas_info.cpp`, particularly the diff part - Using a new class `FFTEngine` to manage discrete Fourier transforms. Plans are re-used as much as possible with different data with the same shape. This is in view of future MPI developments - Redirecting I/O streams in solve functions so they can be used from Python (e.g. in Jupyter notebooks) - Calling `initialize()` and `finalize()` is no longer necessary ### Fixed - Convergence issue with non-linear solvers - Memory error in volume potentials ## v2.1.1 -- 2020-04-22 ### Added - SCons shortcut to run tests ### Fixed - Correct `RPATH` for shared libraries - Issues with SCons commands introduced in v2.1.0 - Tests with Python 2.7 ## v2.1.0 -- 2020-04-17 ### Added - SCons shortcuts to build/install Tamaas and its components - Selection of integration method for Kelvin operator - Compilation option to remove the legacy part of Tamaas - NetCDF dumper ### Fixed - Link bug with clang - NaNs in Kato saturated solver ## v2.0.0 -- 2019-11-11 First public release. Contains relatively mature elastic-plastic contact code. diff --git a/SConstruct b/SConstruct index 8e168ca..3f5f45d 100644 --- a/SConstruct +++ b/SConstruct @@ -1,464 +1,468 @@ # -*- mode:python; coding: utf-8 -*- # vim: set ft=python: # @file # @section LICENSE # # Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), # Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published # by the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . # ------------------------------------------------------------------------------ # Imports # ------------------------------------------------------------------------------ from __future__ import print_function +import sys import os from subprocess import check_output # Import below not strictly necessary, but good for pep8 from SCons.Script import ( EnsurePythonVersion, EnsureSConsVersion, Help, Environment, Variables, EnumVariable, PathVariable, BoolVariable, Split, Export, Dir, ) from SCons.Errors import StopError from SCons import __version__ as scons_version from version import get_git_subst from detect import ( FindFFTW, FindBoost, FindThrust, FindCuda, FindExpolit, FindPybind11 ) from INFOS import TAMAAS_INFOS # ------------------------------------------------------------------------------ EnsurePythonVersion(2, 7) EnsureSConsVersion(2, 4) # ------------------------------------------------------------------------------ def detect_dependencies(env): "Detect all dependencies" fftw_comp = { 'omp': ['omp'], 'threads': ['threads'], 'none': [], } fftw_components = fftw_comp[env['fftw_threads']] if main_env['use_mpi']: fftw_components.append('mpi') FindFFTW(env, fftw_components, precision=env['real_type']) FindBoost(env, ['boost/preprocessor/seq.hpp']) FindExpolit(env) thrust_var = 'THRUST_ROOT' # Take cuda version of thrust if available if 'CUDA_ROOT' in env['ENV']: thrust_var = 'CUDA_ROOT' FindThrust(env, env['backend'], thrust_var) # Activate cuda if needed if env['backend'] == 'cuda': FindCuda(env) if env['build_python']: FindPybind11(env) def subdir(env, dir): "Building a sub-directory" return env.SConscript(env.File('SConscript', dir), variant_dir=env.Dir(dir, env['build_dir']), duplicate=True) # ------------------------------------------------------------------------------ # Main compilation # ------------------------------------------------------------------------------ # Compilation colors colors = { 'cyan': '\033[96m', 'purple': '\033[95m', 'blue': '\033[94m', 'green': '\033[92m', 'yellow': '\033[93m', 'gray': '\033[38;5;8m', 'orange': '\033[38;5;208m', 'red': '\033[91m', 'end': '\033[0m' } # Inherit all environment variables (for CXX detection, etc.) main_env = Environment( ENV=os.environ, ) # Set tamaas information for k, v in TAMAAS_INFOS.items(): main_env[k] = v main_env['COLOR_DICT'] = colors main_env.AddMethod(subdir, 'SubDirectory') # Build variables vars = Variables('build-setup.conf') vars.AddVariables( EnumVariable('build_type', 'Build type', 'release', allowed_values=('release', 'profiling', 'debug'), ignorecase=2), EnumVariable('backend', 'Thrust backend', 'omp', allowed_values=('cpp', 'omp', 'tbb'), ignorecase=2), EnumVariable('fftw_threads', 'Threads FFTW library preference', 'omp', allowed_values=('omp', 'threads', 'none'), ignorecase=2), EnumVariable('sanitizer', 'Sanitizer type', 'none', allowed_values=('none', 'memory', 'leaks', 'address'), ignorecase=2), PathVariable('prefix', 'Prefix where to install', '/usr/local'), # Dependencies paths PathVariable('FFTW_ROOT', 'FFTW custom path', os.getenv('FFTW_ROOT', ''), PathVariable.PathAccept), PathVariable('THRUST_ROOT', 'Thrust custom path', os.getenv('THRUST_ROOT', ''), PathVariable.PathAccept), PathVariable('BOOST_ROOT', 'Boost custom path', os.getenv('BOOST_ROOT', ''), PathVariable.PathAccept), PathVariable('CUDA_ROOT', 'Cuda custom path', os.getenv('CUDA_ROOT', ''), PathVariable.PathAccept), # Dependencies provided as submodule get different default PathVariable('GTEST_ROOT', 'Googletest custom path', os.getenv('GTEST_ROOT', '#third-party/googletest/googletest'), PathVariable.PathAccept), PathVariable('PYBIND11_ROOT', 'Pybind11 custom path', os.getenv('PYBIND11_ROOT', '#third-party/pybind11/include'), PathVariable.PathAccept), PathVariable('EXPOLIT_ROOT', 'Expolit custom path', os.getenv('EXPOLIT_ROOT', '#third-party/expolit/include'), PathVariable.PathAccept), # Executables ('CXX', 'Compiler', os.getenv('CXX', 'g++')), ('MPICXX', 'MPI Compiler wrapper', os.getenv('MPICXX', 'mpicxx')), ('py_exec', 'Python executable', 'python3'), # Compiler flags ('CXXFLAGS', 'C++ compiler flags', os.getenv('CXXFLAGS', "")), # Cosmetic BoolVariable('verbose', 'Activate verbosity', False), BoolVariable('color', 'Color the non-verbose compilation output', False), # Tamaas components BoolVariable('build_doc', 'Build documentation', False), BoolVariable('build_tests', 'Build test suite', False), BoolVariable('build_python', 'Build python wrapper', True), # Dependencies BoolVariable('use_googletest', 'Build tests using GTest', False), BoolVariable('use_mpi', 'Builds multi-process parallelism', False), # Distribution options BoolVariable('strip_info', 'Strip binary of added information', False), BoolVariable('build_static_lib', "Build a static libTamaas", False), # Type variables EnumVariable('real_type', 'Type for real precision variables', 'double', allowed_values=('double', 'long double')), EnumVariable('integer_type', 'Type for integer variables', 'int', allowed_values=('int', 'long')), ) # Set variables of environment vars.Update(main_env) help_text = vars.GenerateHelpText(main_env) help_text += """ Commands: scons [build] [options]... Compile Tamaas (and additional modules/tests) scons install [prefix=/your/prefix] [options]... Install Tamaas to prefix scons dev Install symlink to Tamaas python module (useful to development purposes) scons test Run tests with pytest scons doc Compile documentation with Doxygen and Sphinx+Breathe scons archive Create a gzipped archive from source """ # noqa Help(help_text) # Save all options, not just those that differ from default with open('build-setup.conf', 'w') as setup: for option in vars.options: setup.write("# " + option.help + "\n") setup.write("{} = '{}'\n".format(option.key, main_env[option.key])) main_env['should_configure'] = \ not main_env.GetOption('clean') and not main_env.GetOption('help') build_type = main_env['build_type'] build_dir = 'build-${build_type}' main_env['build_dir'] = main_env.Dir(build_dir) # Printing some build infos if main_env['should_configure']: - print('-- SCons {}'.format(scons_version)) + print('-- SCons {} (Python {}.{})'.format(scons_version, + sys.version_info.major, + sys.version_info.minor)) print(main_env.subst("-- Build type: ${build_type}\n" "-- Thrust backend: ${backend}\n" "-- FFTW threads: ${fftw_threads}\n" "-- MPI: ${use_mpi}\n" "-- Build directory: ${build_dir}")) # Setting up the python name with version if main_env['build_python']: args = (main_env.subst("${py_exec} -c").split() + ["from distutils.sysconfig import get_python_version;" "print(get_python_version())"]) main_env['py_version'] = bytes(check_output(args)).decode() - print(main_env.subst('-- Python version: $py_version')) + print(main_env.subst('-- Python version (bindings): $py_version')) verbose = main_env['verbose'] # Remove colors if not set if not main_env['color']: for key in colors: colors[key] = '' if not verbose: main_env['CXXCOMSTR'] = main_env['SHCXXCOMSTR'] = \ u'{0}[Compiling ($SHCXX)] {1}$SOURCE'.format(colors['green'], colors['end']) main_env['LINKCOMSTR'] = main_env['SHLINKCOMSTR'] = \ u'{0}[Linking] {1}$TARGET'.format(colors['purple'], colors['end']) main_env['ARCOMSTR'] = u'{}[Ar]{} $TARGET'.format(colors['purple'], colors['end']) main_env['RANLIBCOMSTR'] = \ u'{}[Randlib]{} $TARGET'.format(colors['purple'], colors['end']) main_env['PRINT_CMD_LINE_FUNC'] = pretty_cmd_print # Include paths main_env.AppendUnique(CPPPATH=['#/src', '#/src/core', '#/src/mpi', '#/src/bem', '#/src/surface', '#/src/python', '#/src/percolation', '#/src/model', '#/src/model/elasto_plastic', '#/src/solvers', '#/src/gpu', '#/python']) # Changing the shared object extension main_env['SHOBJSUFFIX'] = '.o' # Back to gcc if cuda is activated if main_env['backend'] == "cuda" and "g++" not in main_env['CXX']: raise StopError('GCC should be used when compiling with CUDA') # OpenMP flags - compiler dependent omp_flags = { "g++": ["-fopenmp"], "clang++": ["-fopenmp"], "icpc": ["-qopenmp"] } def cxx_alias(cxx): for k in omp_flags.keys(): if k in cxx: return k raise StopError('Unsupported compiler: ' + cxx) cxx = cxx_alias(main_env['CXX']) # Setting main compilation flags main_env['CXXFLAGS'] = Split(main_env['CXXFLAGS']) main_env['LINKFLAGS'] = main_env['CXXFLAGS'] main_env.AppendUnique( CXXFLAGS=Split('-std=c++14 -Wall -Wextra -pedantic'), - CPPDEFINES=['TAMAAS_BACKEND=TAMAAS_BACKEND_{}'.format( - main_env['backend'].upper() - )], + CPPDEFINES={ + 'TAMAAS_LOOP_BACKEND': 'TAMAAS_LOOP_BACKEND_${backend.upper()}', + 'TAMAAS_FFTW_BACKEND': 'TAMAAS_FFTW_BACKEND_${fftw_threads.upper()}' + }, ) # Adding OpenMP flags if main_env['backend'] == 'omp': main_env.AppendUnique(CXXFLAGS=omp_flags[cxx]) main_env.AppendUnique(LINKFLAGS=omp_flags[cxx]) else: main_env.AppendUnique(CXXFLAGS=['-Wno-unknown-pragmas']) # Correct bug in clang? if main_env['backend'] == 'omp' and cxx == "clang++": main_env.AppendUnique(LIBS=["atomic"]) elif main_env['backend'] == 'tbb': main_env.AppendUnique(LIBS=['tbb']) # Manage MPI compiler if main_env['use_mpi']: main_env['CXX'] = '$MPICXX' main_env.AppendUnique(CPPDEFINES=['TAMAAS_USE_MPI']) main_env.AppendUnique(CXXFLAGS=['-Wno-cast-function-type']) # Flags and options if main_env['build_type'] == 'debug': main_env.AppendUnique(CPPDEFINES=['TAMAAS_DEBUG']) # Define the scalar types main_env.AppendUnique(CPPDEFINES={'TAMAAS_REAL_TYPE': '${real_type}', 'TAMAAS_INT_TYPE': '${integer_type}'}) # Compilation flags cxxflags_dict = { "debug": Split("-g -O0"), "profiling": Split("-g -O3 -fno-omit-frame-pointer"), "release": Split("-O3") } if main_env['sanitizer'] != 'none': if main_env['backend'] == 'cuda': raise StopError( "Sanitizers with cuda are not yet supported!") cxxflags_dict[build_type].append('-fsanitize=${sanitizer}') main_env.AppendUnique(CXXFLAGS=cxxflags_dict[build_type]) main_env.AppendUnique(SHLINKFLAGS=cxxflags_dict[build_type]) main_env.AppendUnique(LINKFLAGS=cxxflags_dict[build_type]) if main_env['should_configure']: basic_checks(main_env) detect_dependencies(main_env) # Writing information file main_env.Tool('textfile') main_env['SUBST_DICT'] = get_git_subst() # Empty values if requested if main_env['strip_info']: for k in main_env['SUBST_DICT']: main_env['SUBST_DICT'][k] = "" # Substitution of environment file main_env['SUBST_DICT'].update({ '@build_type@': '$build_type', '@build_dir@': '${build_dir.abspath}', '@build_version@': '$version', }) # Environment file content env_content = """export PYTHONPATH=@build_dir@/python:$$PYTHONPATH export LD_LIBRARY_PATH=@build_dir@/src:$$LD_LIBRARY_PATH """ # Writing environment file env_file = main_env.Textfile( main_env.File('tamaas_environment.sh', main_env['build_dir']), env_content) # Default targets build_targets = ['build-cpp', env_file] install_targets = ['install-lib'] if main_env._get_major_minor_revision(scons_version)[0] >= 4: main_env.Tool('compilation_db') main_env.CompilationDatabase(PRINT_CMD_LINE_FUNC=pretty_cmd_print) # Building Tamaas library Export('main_env') main_env.SubDirectory('src') # Building Tamaas extra components for dir in ['python', 'tests']: if main_env['build_{}'.format(dir)] and not main_env.GetOption('help'): main_env.SubDirectory(dir) build_targets.append('build-{}'.format(dir)) # Building API + Sphinx documentation if requested if main_env['build_doc']: main_env.SubDirectory('doc') main_env.Alias('doc', 'build-doc') install_targets.append('install-doc') else: dummy_command(main_env, 'doc', 'Command "doc" does not do anything' ' without documentation activated ("build_doc=True")') # Define dummy dev command when python is deactivated if not main_env['build_python']: dummy_command(main_env, 'dev', 'Command "dev" does not do anything' + ' without python activated ("build_python=True")') else: install_targets.append('install-python') # Define dummy test command when tests are deactivated if not main_env['build_tests']: dummy_command(main_env, 'test', 'Command "test" does not do anything' + ' without tests activated ("build_tests=True")') # Definition of target aliases, a.k.a. sub-commands main_env.Alias('build', build_targets) # Define proper install targets main_env.Alias('install', install_targets) # Default target is to build stuff main_env.Default('build') # Building a tar archive archive = main_env.Command( 'tamaas-{}.tar.gz'.format(main_env['version']), '', ('tar --exclude-vcs --exclude-vcs-ignores ' '--exclude=third-party/googletest ' '--exclude=third-party/pybind11 ' '--exclude=joss ' '--exclude=".*" ' '-czf $TARGET {}'.format(Dir('.').name)), chdir='..', ) main_env.Alias('archive', archive) diff --git a/site_scons/site_init.py b/site_scons/site_init.py index 94345ed..7260287 100644 --- a/site_scons/site_init.py +++ b/site_scons/site_init.py @@ -1,178 +1,179 @@ # -*- coding: utf-8 -*- # @file # @section LICENSE # # Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), # Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published # by the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . import subprocess from detect import FindPybind11 from SCons.Script import Configure from SCons.Errors import StopError # ------------------------------------------------------------------------------ def pybind11(env): """A tool to configure pybind11""" def execute(command): return [line for line in subprocess.check_output( command, universal_newlines=True).split('\n') if line != ""] # Create a clone so we do not modify env clone = env.Clone() # Set variables for clone FindPybind11(clone) includes = clone['CPPPATH'] # Extension of shared library for python try: extension = execute( '{}-config --extension-suffix'.format(env['py_exec']).split())[0] except subprocess.CalledProcessError: extension = ".so" def pybind11_builder(env, target, source, **kwargs): """Create a pybind11 module""" clone = env.Clone() clone.AppendUnique(CPPPATH=includes) clone['SHLIBSUFFIX'] = extension clone.AppendUnique(SHLINKFLAGS=['-fvisibility=hidden']) return clone.SharedLibrary(target, source, **kwargs) # Add pseudo-builder to master environment env.AddMethod(pybind11_builder, 'Pybind11Module') # ------------------------------------------------------------------------------ def pretty_cmd_print(command, target, source, env): colors = env['COLOR_DICT'] if 'Copy' in command: color = colors['gray'] action = 'Copying' elif 'Creating' in command: color = colors['yellow'] action = 'Generating' elif 'database' in command: color = colors['yellow'] action = 'Generating' else: print(command) return print("{color}[{action}] {end}{target}".format( color=color, end=colors['end'], target=target[0], action=action )) # ------------------------------------------------------------------------------ def CheckPythonModule(context, module): """Checks the existence of a python module""" context.Message('Checking for Python module {}... '.format(module)) env = context.sconf.env command = [env.subst('${py_exec}'), '-c', 'import {}'.format(module)] context.Log('Executing external command: {}\n'.format(command)) try: subprocess.check_output(command, stderr=subprocess.STDOUT) result = True except subprocess.CalledProcessError as e: result = False output = bytes(e.output) context.Log(output.decode() + '\n') context.Result(result) return result # ------------------------------------------------------------------------------ def CheckCompilerFlag(context, flag): "Check compiler provides flag" - context.Message('Checking compiler flag {}... '.format(flag)) + context.Message('Checking compiler flag {}... '.format( + context.sconf.env.subst(flag))) test_file = """ int main() { return 0; } """ env = context.sconf.env.Clone(CXXFLAGS=[flag], LINKFLAGS=[flag]) context.sconf.env = env result = context.TryLink(test_file, '.cpp') context.Result(result) return result # ------------------------------------------------------------------------------ def dummy_command(env, command, error_msg): """Creates a dummy scons command""" def print_error(*args, **kwargs): print(error_msg) def print_cmd(*args, **kwargs): pass comm = env.Command('#.phony_{}'.format(command), '', print_error, PRINT_CMD_LINE_FUNC=print_cmd) env.Alias(command, comm) # ------------------------------------------------------------------------------ def get_python_version(env): versions_script = """ from __future__ import print_function from sysconfig import get_python_version print(get_python_version())""" version = subprocess.check_output([env['py_exec'], "-c", versions_script], universal_newlines=True).replace('\n', '') print(version) return version # ------------------------------------------------------------------------------ def basic_checks(env): custom_tests = { 'CheckCompilerFlag': CheckCompilerFlag, } conf = Configure(env, custom_tests=custom_tests) if not conf.CheckCXX(): StopError('Could not find working C++ compiler') for flag in env['CXXFLAGS']: if not conf.CheckCompilerFlag(flag): StopError('Compiler does not support flag {}'.format(flag)) def check_type(cpp_type, stl_header): if not conf.CheckType(cpp_type, '#include <{}>'.format(stl_header), 'cpp'): StopError('Standard type {} is not available'.format(cpp_type)) check_type('std::multiplies', 'functional') check_type('std::unique_ptr', 'memory') conf.Finish() diff --git a/src/core/loop.hh b/src/core/loop.hh index 9cb7579..6fc2b7e 100644 --- a/src/core/loop.hh +++ b/src/core/loop.hh @@ -1,242 +1,242 @@ /** * @file * @section LICENSE * * Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), * Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) * * This program is free software: you can redistribute it and/or modify * it under the terms of the GNU Affero General Public License as published * by the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU Affero General Public License for more details. * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see . * */ /* -------------------------------------------------------------------------- */ #ifndef LOOP_HH #define LOOP_HH /* -------------------------------------------------------------------------- */ #include "loops/apply.hh" #include "loops/loop_utils.hh" #include "mpi_interface.hh" #include "ranges.hh" #include "tamaas.hh" #include #include #include #include #include #include #include #include #include namespace tamaas { template struct is_policy : std::false_type {}; template <> struct is_policy : std::true_type {}; template <> struct is_policy : std::true_type {}; template <> struct is_policy : std::true_type {}; // device_t == host_t when thrust backend is CPP -#if TAMAAS_BACKEND != TAMAAS_BACKEND_CPP +#if TAMAAS_LOOP_BACKEND != TAMAAS_LOOP_BACKEND_CPP template <> struct is_policy : std::true_type {}; template <> struct is_policy : std::true_type {}; template <> struct is_policy : std::true_type {}; #endif /** * @brief Singleton class for automated loops using lambdas * This class is sweet candy :) It provides abstraction of the paralelism * paradigm used in loops and allows simple and less error-prone loop syntax, * with minimum boiler plate. I love it <3 */ class Loop { public: /// Backends enumeration enum backend { omp, ///< [OpenMP](http://www.openmp.org/specifications/) backend cuda, ///< [Cuda](http://docs.nvidia.com/cuda/index.html) backend }; /// Helper class to count iterations within lambda-loop template class arange { public: using it_type = thrust::counting_iterator; using reference = typename it_type::reference; arange(T start, T size) : start(start), range_size(size) {} it_type begin(UInt /*i*/ = 1) const { return it_type(T(start)); } it_type end(UInt /*i*/ = 1) const { return it_type(range_size); } UInt getNbComponents() const { return 1; } private: T start, range_size; }; template static arange range(T size) { return arange(0, size); } template static arange range(U start, T size) { return arange(start, size); } private: /// Replacement for thrust::transform_iterator which copies values template class transform_iterator : public thrust::iterator_adaptor< transform_iterator, Iterator, Value, thrust::use_default, thrust::use_default, Value> { Functor func; public: using super_t = thrust::iterator_adaptor, Iterator, Value, thrust::use_default, thrust::use_default, Value>; __host__ __device__ transform_iterator(const Iterator& x, const Functor& func) : super_t(x), func(func) {} friend class thrust::iterator_core_access; private: __host__ __device__ auto dereference() const -> decltype(func(*this->base())) { return func(*this->base()); } }; public: /// Loop functor over ranges template static auto loop(Functor&& func, Ranges&&... ranges) -> typename std::enable_if::value, void>::type { loopImpl(thrust::device, std::forward(func), std::forward(ranges)...); } /// Loop over ranges with non-default policy template static void loop(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges) { loopImpl(policy, std::forward(func), std::forward(ranges)...); } private: template using reference_type = typename std::decay::type::reference; public: /// Reduce functor over ranges template static auto reduce(Functor&& func, Ranges&&... ranges) -> typename std::enable_if< not is_policy::value, decltype(func(std::declval>()...))>::type { return reduceImpl(thrust::device, std::forward(func), std::forward(ranges)...); } /// Reduce over ranges with non-default policy template static auto reduce(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges) -> decltype(func(std::declval>()...)) { return reduceImpl(policy, std::forward(func), std::forward(ranges)...); } private: /// Loop over ranges and apply functor template static void loopImpl(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges); /// Loop over ranges, apply functor and reduce result template static auto reduceImpl(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges) -> decltype(func(std::declval>()...)); public: /// Constructor Loop() = delete; }; /* -------------------------------------------------------------------------- */ /* Template implementation */ /* -------------------------------------------------------------------------- */ template void Loop::loopImpl(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges) { auto begin = thrust::make_zip_iterator(thrust::make_tuple(ranges.begin()...)); auto end = thrust::make_zip_iterator(thrust::make_tuple(ranges.end()...)); checkLoopSize(ranges...); thrust::for_each(policy, begin, end, detail::ApplyFunctor(std::forward(func))); #if (defined(TAMAAS_USE_CUDA) && THRUST_VERSION < 100904) cudaDeviceSynchronize(); #endif } /* -------------------------------------------------------------------------- */ template auto Loop::reduceImpl(const thrust::execution_policy& policy, Functor&& func, Ranges&&... ranges) -> decltype(func(std::declval>()...)) { using return_type = decltype(func(std::declval>()...)); using apply_type = detail::ApplyFunctor; checkLoopSize(ranges...); auto applier = apply_type(func); detail::reduction_helper red_helper; auto begin_zip = thrust::make_zip_iterator(thrust::make_tuple(ranges.begin()...)); auto end_zip = thrust::make_zip_iterator(thrust::make_tuple(ranges.end()...)); transform_iterator begin( begin_zip, applier); transform_iterator end(end_zip, applier); auto result = thrust::reduce(policy, begin, end, red_helper.init(), red_helper); #if (defined(TAMAAS_USE_CUDA) && THRUST_VERSION < 100904) cudaDeviceSynchronize(); #endif return mpi::allreduce(std::move(result)); } } // namespace tamaas #endif // LOOP_HH diff --git a/src/core/tamaas.cpp b/src/core/tamaas.cpp index 18333b9..6dfb9f9 100644 --- a/src/core/tamaas.cpp +++ b/src/core/tamaas.cpp @@ -1,101 +1,100 @@ /** * @file * @section LICENSE * * Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), * Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) * * This program is free software: you can redistribute it and/or modify * it under the terms of the GNU Affero General Public License as published * by the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU Affero General Public License for more details. * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see . * */ /* -------------------------------------------------------------------------- */ #include "tamaas.hh" #include "fftw_interface.hh" #include "fftw_mpi_interface.hh" #include "logger.hh" #include "mpi_interface.hh" -#if TAMAAS_BACKEND == TAMAAS_BACKEND_OMP +#if TAMAAS_LOOP_BACKEND == TAMAAS_LOOP_BACKEND_OMP #include #endif /* -------------------------------------------------------------------------- */ namespace tamaas { void initialize(UInt num_threads) { static bool has_warned = false; mpi::thread provided = mpi::thread::single; if (not mpi::initialized()) mpi::init_thread(nullptr, nullptr, mpi::thread::single, &provided); bool should_init_threads = (provided > mpi::thread::single); - if (num_threads) { -#if TAMAAS_BACKEND == TAMAAS_BACKEND_OMP +#if TAMAAS_LOOP_BACKEND == TAMAAS_LOOP_BACKEND_OMP + if (num_threads) omp_set_num_threads(num_threads); // set user-defined number of threads -#endif - } else { + else + num_threads = omp_get_max_threads(); +#else + if (num_threads != 0) num_threads = 1; - } +#endif -#if TAMAAS_BACKEND != TAMAAS_BACKEND_CPP - if (should_init_threads and (not fftw::init_threads())) { +#if TAMAAS_FFTW_BACKEND != TAMAAS_FFTW_BACKEND_NONE + if (should_init_threads and (not fftw::init_threads())) TAMAAS_EXCEPTION("FFTW could not initialize threads!"); - } #endif if (mpi::initialized()) { if (not has_warned) { Logger().get(LogLevel::warning) << "WARNING: experimental MPI support\n"; has_warned = true; } fftw::mpi::init(); } if (should_init_threads) { -#if TAMAAS_BACKEND == TAMAAS_BACKEND_OMP - fftw::plan_with_nthreads(omp_get_max_threads()); -#elif TAMAAS_BACKEND == TAMAAS_BACKEND_TBB +#if TAMAAS_FFTW_BACKEND != TAMAAS_FFTW_BACKEND_NONE fftw::plan_with_nthreads(num_threads); #endif } } /* -------------------------------------------------------------------------- */ void finalize() { if (not mpi::finalized()) { #if TAMAAS_BACKEND != TAMAAS_BACKEND_CPP fftw::cleanup_threads(); #endif fftw::mpi::cleanup(); mpi::finalize(); } } namespace { /// Manager for initialize + finalize struct entry_exit_points { entry_exit_points() { initialize(); } ~entry_exit_points() { finalize(); } static const entry_exit_points singleton; }; const entry_exit_points entry_exit_points::singleton; } // namespace } // namespace tamaas diff --git a/src/core/tamaas.hh b/src/core/tamaas.hh index 7300085..c6ba56a 100644 --- a/src/core/tamaas.hh +++ b/src/core/tamaas.hh @@ -1,172 +1,180 @@ /** * @mainpage Tamaas - A high-performance periodic contact library * * @section Introduction * Tamaas is a spectral-integral-equation based contact library. It is made * with love to be fast and friendly! * * @author Guillaume Anciaux * @author Lucas Frérot * @author Valentine Rey * @author Son Pham-Ba * * @section LICENSE * * Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), * Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) * * This program is free software: you can redistribute it and/or modify * it under the terms of the GNU Affero General Public License as published * by the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU Affero General Public License for more details. * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see . * */ /* -------------------------------------------------------------------------- */ #ifndef TAMAAS_HH #define TAMAAS_HH /* -------------------------------------------------------------------------- */ -#define TAMAAS_BACKEND_OMP 1 -#define TAMAAS_BACKEND_TBB 2 -#define TAMAAS_BACKEND_CPP 3 -#ifndef TAMAAS_BACKEND -#define TAMAAS_BACKEND TAMAAS_BACKEND_OMP +#define TAMAAS_FFTW_BACKEND_OMP 2 +#define TAMAAS_FFTW_BACKEND_THREADS 2 +#define TAMAAS_FFTW_BACKEND_NONE 3 +#define TAMAAS_LOOP_BACKEND_OMP 1 +#define TAMAAS_LOOP_BACKEND_TBB 2 +#define TAMAAS_LOOP_BACKEND_CPP 3 +// Default loop backend is OpenMP +#ifndef TAMAAS_LOOP_BACKEND +#define TAMAAS_LOOP_BACKEND TAMAAS_LOOP_BACKEND_OMP +#endif +// Default FFTW backend is none +#ifndef TAMAAS_FFTW_BACKEND +#define TAMAAS_FFTW_BACKEND TAMAAS_FFTW_BACKEND_NONE #endif // If the thrust device hasn't been set, set OpenMP #ifndef THRUST_DEVICE_SYSTEM #define THRUST_DEVICE_SYSTEM THRUST_DEVICE_SYSTEM_OMP #endif /* -------------------------------------------------------------------------- */ // Standard includes #include #include #include #include #include /* -------------------------------------------------------------------------- */ // Special thrust includes #include #include #ifdef TAMAAS_USE_CUDA #include "unified_allocator.hh" #endif #include "fftw_allocator.hh" /* -------------------------------------------------------------------------- */ namespace tamaas { /* -------------------------------------------------------------------------- */ /// @section Cuda specific definitions #define CUDA_LAMBDA __device__ __host__ #ifdef TAMAAS_USE_CUDA template using Allocator = UnifiedAllocator; #else template using Allocator = FFTWAllocator; #endif /// @section Common types definitions // If type macros have not been set, put default values #ifndef TAMAAS_REAL_TYPE #define TAMAAS_REAL_TYPE double #endif #ifndef TAMAAS_INT_TYPE #define TAMAAS_INT_TYPE int #endif using Real = TAMAAS_REAL_TYPE; ///< default floating point type using Int = TAMAAS_INT_TYPE; ///< default signed integer type using UInt = std::make_unsigned_t; ///< default unsigned integer type template using complex = thrust::complex; ///< template complex wrapper using Complex = complex; ///< default floating point complex type /// @section Defining random toolbox using ::thrust::random::normal_distribution; using ::thrust::random::uniform_real_distribution; using random_engine = ::thrust::random::default_random_engine; namespace detail { template class Trait, typename Head, typename... Tail> struct fold_trait_tail_rec : std::integral_constant::value, Trait, Tail...>::value> {}; template class Trait, typename Head> struct fold_trait_tail_rec : std::integral_constant::value> {}; } // namespace detail template