diff --git a/CHANGELOG.md b/CHANGELOG.md index e6f848e..98d8324 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,231 +1,233 @@ # Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) for final versions and [PEP440](https://www.python.org/dev/peps/pep-0440/) in case intermediate versions need to be released (e.g. development version `2.2.3.dev1` or release candidates `2.2.3rc1`), or individual commits are packaged. ## Unreleased ### Changed - The root `Dockerfile` now compiles Tamaas, so using Tamaas in Docker is easier. - The model attributes dumped to Numpy files are now written in a JSON-formatted string to avoid unsafe loading/unpickling of objects. - Removed the `build_doc` build option: now the doc targets are automatically added if the dependencies are met, and built if `scons doc` is called. +- Removed the `use_googletest` build option: if tests are built and gtest is + present, the corresponding tests will be built ## v2.3.0 -- 2021-06-15 ### Added - Added `read()` method to dumpers to create a model from a dump file - `getClusters()` can be called in MPI contact with partial contact maps - Added a JSON encoder class for models and a JSON dumper - CUDA compatibility is re-established, but has not been tested - Docstrings in the Python bindings for many classes/methods ### Changed - Tamaas version numbers are now managed by [versioneer](https://github.com/python-versioneer/python-versioneer). This means that Git tags prefixed with `v` (e.g. `v2.2.3`) carry meaning and determine the version. When no tag is set, versioneer uses the last tag, specifies the commit short hash and the distance to the last tag (e.g. `2.2.2+33.ge314b0e`). This version string is used in the compiled library, the `setup.py` script and the `__version__` variable in the python module. - Tamaas migrated to [GitLab](https://gitlab.com/tamaas/tamaas) - Continuous delivery has been implemented: - the `master` branch will now automatically build and publish Python wheels to `https://gitlab.com/api/v4/projects/19913787/packages/pypi/simple`. These "nightly" builds can be installed with: pip install \ --extra-index-url https://gitlab.com/api/v4/projects/19913787/packages/pypi/simple \ tamaas - version tags pushed to `master` will automatically publish the wheels to [PyPI](https://pypi.org/project/tamaas/) ### Deprecated - The `finalize()` function is now deprecated, since it is automatically called when the process terminates - Python versions 3.5 and below are not supported anymore ### Fixed - Fixed a host of dump read/write issues when model type was not `volume_*d`. Dumper tests are now streamlined and systematic. - Fixed a bug where `Model::solveDirichlet` would not compute correctly - Fixed a bug where `Statistics::contact` would not normalize by the global number of surface points ## v2.2.2 -- 2021-04-02 ### Added - Entry-point `tamaas` defines a grouped CLI for `examples/pipe_tools`. Try executing `tamaas surface -h` from the command-line! ### Changed - `CXXFLAGS` are now passed to the linker - Added this changelog - Using absolute paths for environmental variables when running `scons test` - Reorganized documentation layout - Gave the build system a facelift (docs are now generated directly with SCons instead of a Makefile) ### Deprecated - Python 2 support is discontinued. Version `v2.2.1` is the last PyPi build with a Python 2 wheel. - The scripts in `examples/pipe_tools` have been replaced by the `tamaas` command ### Fixed - `UVWDumper` no longer imports `mpi4py` in sequential - Compiling with different Thrust/FFTW backends ## v2.2.1 -- 2021-03-02 ### Added - Output registered fields and dumpers in `print(model)` - Added `operator[]` to the C++ model class (for fields) - Added `traction` and `displacement` properties to Python model bindings - Added `operators` property to Python model bindings, which provides a dict-like access to registered operators - Added `shape` and `spectrum` to properties to Python surface generator bindings - Surface generator constructor accepts surface global shape as argument - Choice of FFTW thread model ### Changed - Tests use `/tmp` for temporary files - Updated dependency versions (Thrust, Pybind11) ### Deprecated - Most `get___()` and `set___()` in Python bindings have been deprecated. They will generate a `DeprecationWarning`. ### Removed - All legacy code ## v2.2.0 -- 2020-12-31 ### Added - More accurate function for computation of contact area - Function to compute deviatoric of tensor fields - MPI implementation - Convenience `hdf5toVTK` function - Readonly properties `shape`, `global_shape`, `boundary_shape` on model to give shape information ### Changed - Preprocessor defined macros are prefixed with `TAMAAS_` - Moved `tamaas.to_voigt` to `tamaas.compute.to_voigt` ### Fixed - Warning about deprecated constructors with recent GCC versions - Wrong computation of grid strides - Wrong computation of grid sizes in views ## v2.1.4 -- 2020-08-07 ### Added - Possibility to generate a static `libTamaas` - C++ implementation of DFSANE solver - Allowing compilation without OpenMP ### Changed - NetCDF dumper writes frames to a single file ### Fixed - Compatibility with SCons+Python 3 ## v2.1.3 -- 2020-07-27 ### Added - Version number to `TamaasInfo` ### Changed - Prepending root directory when generating archive ## v2.1.2 -- 2020-07-24 This release changes some core internals related to discrete Fourier transforms for future MPI support. ### Added - Caching `CXXFLAGS` in SCons build - SCons shortcut to create code archive - Test of the elastic-plastic contact solver - Paraview data dumper (`.pvd` files) - Compression for UVW dumper - `__contains__` and `__iter__` Python bindings of model - Warning message of possible overflow in Kelvin ### Changed - Simplified `tamaas_info.cpp`, particularly the diff part - Using a new class `FFTEngine` to manage discrete Fourier transforms. Plans are re-used as much as possible with different data with the same shape. This is in view of future MPI developments - Redirecting I/O streams in solve functions so they can be used from Python (e.g. in Jupyter notebooks) - Calling `initialize()` and `finalize()` is no longer necessary ### Fixed - Convergence issue with non-linear solvers - Memory error in volume potentials ## v2.1.1 -- 2020-04-22 ### Added - SCons shortcut to run tests ### Fixed - Correct `RPATH` for shared libraries - Issues with SCons commands introduced in v2.1.0 - Tests with Python 2.7 ## v2.1.0 -- 2020-04-17 ### Added - SCons shortcuts to build/install Tamaas and its components - Selection of integration method for Kelvin operator - Compilation option to remove the legacy part of Tamaas - NetCDF dumper ### Fixed - Link bug with clang - NaNs in Kato saturated solver ## v2.0.0 -- 2019-11-11 First public release. Contains relatively mature elastic-plastic contact code. diff --git a/Jenkinsfile b/Jenkinsfile index 369ed3d..75a8de1 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -1,131 +1,130 @@ pipeline { parameters {string(defaultValue: '', description: 'api-token', name: 'API_TOKEN') string(defaultValue: '', description: 'Token for readthedocs', name: 'RTD_TOKEN') string(defaultValue: '', description: 'buildable phid', name: 'BUILD_TARGET_PHID') string(defaultValue: '', description: 'Commit id', name: 'COMMIT_ID') string(defaultValue: '', description: 'Diff id', name: 'DIFF_ID') string(defaultValue: 'PHID-PROJ-gbo56hpf2y5bi7t5jusk', description: 'ID of the project', name: 'PROJECT_ID') } environment { PHABRICATOR_HOST = 'https://c4science.ch/api/' PYTHONPATH = sh returnStdout: true, script: 'echo ${WORKSPACE}/build-release/python:${WORKSPACE}/tests/ci/script/' } agent {dockerfile {additionalBuildArgs '--tag tamaas-environment'}} options {timeout(time: 1, unit: 'HOURS')} stages { stage('SCM Checkout') { steps { checkout scm: [ $class: 'GitSCM', branches: scm.branches, extensions: [[ $class: 'SubmoduleOption', recursiveSubmodules: true, ]], userRemoteConfigs: scm.userRemoteConfigs ] } } stage('Configure') { steps { writeFile(file: build-setup.conf, text:'''py_exec = \'python3\' build_python = \'true\' build_tests = \'true\' - use_googletest = \'true\' use_mpi = \'true\' verbose = \'true\'"''') } } stage('Compile') { steps { tee('compilation.log') { sh '''set -o pipefail rm -rf .sconf_temp .sconsign.dblite build-release scons''' } } post { always { uploadArtifact('config.log', 'Configure log') uploadArtifact('compilation.log', 'Compilation log') } } } stage('Run tests') { steps { tee('tests.log') { sh '''set -o pipefail python3 -m pytest -vvv --durations=0 --junitxml=results.xml build-release/tests''' } } post { always { uploadArtifact('tests.log', 'Tests log') createArtifact("results.xml") junit 'results.xml' step([$class: 'XUnitBuilder', thresholds: [ [$class: 'SkippedThreshold', failureThreshold: '0'], [$class: 'FailedThreshold', failureThreshold: '0']], tools: [[$class: 'GoogleTestType', pattern: 'build-release/tests/gtest_results.xml']]]) } } } } post { success { trigger_rtd() passed_hbm() } failure { failed_hbm() emailext( body: '''${SCRIPT, template="groovy-html.template"}''', mimeType: 'text/html', subject: "[Jenkins] ${currentBuild.fullDisplayName} Failed", recipientProviders: [[$class: 'CulpritsRecipientProvider']], to: 'lfrerot1@jhu.edu', attachLog: true, compressLog: true) } } } def createArtifact(filename) { sh "./tests/ci/scripts/hbm send-uri -k 'Jenkins URI' -u ${BUILD_URL} -l 'View Jenkins result'" sh "./tests/ci/scripts/hbm send-junit-results -f ${filename}" } def uploadArtifact(artifact, name) { archiveArtifacts(artifacts: "${artifact}", fingerprint: true) sh "./tests/ci/scripts/hbm upload-file -f ${artifact} -n \"${name}\" -v ${PROJECT_ID}" } def failed_hbm() { sh "./tests/ci/scripts/hbm failed" } def passed_hbm() { sh "./tests/ci/scripts/hbm passed" } def trigger_rtd() { def response = httpRequest(url: "https://readthedocs.org/api/v2/webhook/tamaas/106141/", httpMode: 'POST', requestBody: "token=${RTD_TOKEN}") println("Status: " + response.status) println("Content: " + response.content) } diff --git a/README.md b/README.md index 58f2136..b245bbe 100644 --- a/README.md +++ b/README.md @@ -1,222 +1,222 @@ Tamaas --- A high-performance library for periodic rough surface contact ======================================================================== Tamaas is a C++/Python library that implements a number of numerical methods based on integral equations to efficiently solve contact problems with rough surfaces. The word تماس (tamaas) means "contact" in Arabic and Farsi. ## Quick Start If you have a Linux system, you can simply run `pip(3) install tamaas`. Note however that there due to compatibility reasons, this version of Tamaas is not built with parallel/multi-threading capabilities. So if you want parallelism, or encounter an issue with the [PyPI package](https://pypi.org/project/tamaas/), please compile from source. You can also use one of our [interactive tutorial notebooks](https://tamaas.readthedocs.io/en/latest/overview.html#tutorials) to get started. ## Dependencies Here is a list of dependencies to compile Tamaas: - a C++ compiler with full C++14 and OpenMP support - [SCons](https://scons.org/) (python build system) - [FFTW3](http://www.fftw.org/) - [boost](https://www.boost.org/) (preprocessor) - [thrust](https://github.com/thrust/thrust) (1.9.2+) - [python 3+](https://www.python.org/) with [numpy](https://numpy.org/) - [pybind11](https://github.com/pybind/pybind11) (included as submodule) Optional dependencies are: - an MPI implmentation - [FFTW3](http://www.fftw.org/) with MPI/pthreads/OpenMP support - [scipy](https://scipy.org) (for nonlinear solvers) - [uvw](https://pypi.org/project/uvw/), [netCDF4](https://unidata.github.io/netcdf4-python/netCDF4/index.html#section1), [h5py](https://www.h5py.org/) (for various dumpers) - [googletest](https://github.com/google/googletest) and [pytest](https://docs.pytest.org/en/latest/) (for tests) - [Doxygen](http://doxygen.nl/) and [Sphinx](https://www.sphinx-doc.org/en/stable/) (for documentation) Note that a Debian distribution should have the right packages for all these dependencies (they package the right version of thrust extracted from CUDA in `stretch-backports non-free` and `buster non-free`). ## Compiling ### Dependencies To simplify compilation, pybind11 and googletest can be setup via git submodules. We still recommend installing them via your package manager. git submodule update --init --recursive ### Build System The build system uses SCons. In order to compile Tamaas with the default options (i.e. optimized release build with python bindings + OpenMP): scons After compiling a first time (or running `scons -h` if you want to avoid compiling), you can edit the compilation options in the file `build-setup.conf`, or alternatively supply the options directly in the command line: scons option=value [...] To get a list of *all* build options and their possible values, you can run `scons -h`. You can run `scons -H` to see the SCons-specific options (among them `-j n` executes the build with `n` threads and `-c` cleans the build). Note that the build is aware of the `CXX` and `CXXFLAGS` environment variables. ## Installing Before you can import tamaas in python, you need to install the python package in some way. ### Using pip You have two choices to install tamaas: - An out-of-repository installation to a given prefix (e.g. `/usr/local`, or a python virtual environment) - A development installation to `~/.local` which links to the build directory The former is simply achieved with: scons prefix=/your/prefix install # Equivalent to (if you build in release) install build-release/src/libTamaas.so* /your/prefix/lib pip3 install --prefix /your/prefix build-release/python The compiled parts of the python module should automatically know where to find the Tamaas shared library, so no need to tinker with `LD_LIBRARY_PATH`. The second installation choice is equally simple: scons dev # Equivalent to pip3 install --user -e build-release/python You can check that everything is working fine with: python3 -c 'import tamaas; print(tamaas)' ### Using environment variables (not recommended) You can source (e.g. in your `~/.bashrc` file) the file `build-release/tamaas_environment.sh` to modify the `PYTHONPATH` and `LD_LIBRARY_PATH` environment variables. This is however not recommended because these variables may conflict in a python virtual environment (i.e. if you use `virtualenv` with tamaas). ### Using Docker image Tamaas provides a `Dockerfile` you can use to compile and deploy Tamaas in a tested environment. Simply run `docker build -t tamaas .` in the root folder of the repository. Check the documentation to see which build options can be adjusted. ## Tests To run tests, make sure to have [pytest](https://docs.pytest.org/en/latest/) installed and run `scons test` if you have compiled Tamaas with tests activated -(`scons build_tests=True use_googletest=True`). +(`scons build_tests=True`). ## Documentation The latest documentation is available on [ReadTheDocs](https://tamaas.readthedocs.io/en/latest/)! It contains detailed guides on the different components of Tamaas as well as a combined C++ and Python API documentation. You can also build the documentation locally: activate the `build_doc` option and run `scons doc`. Make sure you have [sphinx-rtd-theme](https://pypi.org/project/sphinx-rtd-theme/) and [breathe](https://pypi.org/project/breathe/) installed. The compiled indexes for the doxygen C++ API and Sphinx documentation can be found in `doc/build/{doxygen,sphinx}/html/index.html`. Beware however that manually compiling documentation leads to a lot of warnings. ## Examples Example simulations can be found in the `examples/` directory. There is no guarantee that the examples in `examples/legacy/` all work however. - `rough_contact.py` shows a typical normal rough contact simulation - `adhesion.py` shows how you can derive some classes from Tamaas in python, here to implement a custom adhesion potential - `plasticity.py` computes an elastoplastic Hertz simulation and dumps the result in `examples/paraview/` in VTK format - `stresses.py` shows how you can compute stresses from a boundary traction distribution - the scripts in `pipe_tools` allow to execute elastic contact simulations without the need to code a custom script (see documentation for more details) ## Command-line Interface The Tamaas python module exposes a `tamaas` command with three subcommands: - `tamaas surface` generates rough surfaces - `tamaas contact` solves elastic contact - `tamaas plot` plots contact pressure and displacement See `tamaas --help` for command line arguments. ## Contributing Contributions to Tamaas are welcome! Please follow the guidelines below. ### Report an issue If you have an account on [gitlab](https://gitlab.com), you can [submit an issue](https://gitlab.com/tamaas/tamaas/-/issues/new). The full list of issues is available [here](https://gitlab.com/tamaas/tamaas/-/issues). ### Submit a patch / merge-request Follow [this guide](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html#new-merge-request-from-a-fork) to create a merge request on GitLab. Please target the repository's `master` branch. ## Citing Tamaas is the result of a science research project. To give proper credit to Tamaas and the researchers who have developed the numerical methods that it implements, please cite Tamaas as: Frérot , L., Anciaux, G., Rey, V., Pham-Ba, S., & Molinari, J.-F. Tamaas: a library for elastic-plastic contact of periodic rough surfaces. Journal of Open Source Software, 5(51), 2121 (2020). [doi:10.21105/joss.02121](https://doi.org/10.21105/joss.02121) If you use the elastic-plastic contact capabilities of Tamaas, please cite: Frérot, L., Bonnet, M., Molinari, J.-F. & Anciaux, G. A Fourier-accelerated volume integral method for elastoplastic contact. Computer Methods in Applied Mechanics and Engineering 351, 951–976 (2019) [doi:10.1016/j.cma.2019.04.006](https://doi.org/10.1016/j.cma.2019.04.006). If you use the adhesive contact capabilities of Tamaas, please cite: Rey, V., Anciaux, G. & Molinari, J.-F. Normal adhesive contact on rough surfaces: efficient algorithm for FFT-based BEM resolution. Comput Mech 1–13 (2017) [doi:10.1007/s00466-017-1392-5](https://doi.org/10.1007/s00466-017-1392-5). ## Changelog The changelog can be consulted [here](https://gitlab.com/tamaas/tamaas/-/blob/master/CHANGELOG.md). ## License Tamaas is distributed under the terms of the [GNU Affero General Public License v3.0](https://www.gnu.org/licenses/agpl.html). diff --git a/SConstruct b/SConstruct index 2d13359..0497e37 100644 --- a/SConstruct +++ b/SConstruct @@ -1,475 +1,474 @@ # -*- mode:python; coding: utf-8 -*- # vim: set ft=python: # # Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), # Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published # by the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . # ------------------------------------------------------------------------------ # Imports # ------------------------------------------------------------------------------ from __future__ import print_function import sys import os from subprocess import check_output # Import below not strictly necessary, but good for pep8 from SCons.Script import ( EnsurePythonVersion, EnsureSConsVersion, Help, Environment, Variables, EnumVariable, PathVariable, BoolVariable, ListVariable, Split, Export, Dir, ) from SCons.Errors import StopError from SCons import __version__ as scons_version from version import get_git_subst from detect import ( FindFFTW, FindBoost, FindThrust, FindCuda, FindExpolit, FindPybind11 ) from INFOS import TAMAAS_INFOS # ------------------------------------------------------------------------------ EnsurePythonVersion(2, 7) EnsureSConsVersion(2, 4) # ------------------------------------------------------------------------------ def detect_dependencies(env): "Detect all dependencies" fftw_comp = { 'omp': ['omp'], 'threads': ['threads'], 'none': [], } fftw_components = fftw_comp[env['fftw_threads']] if main_env['use_mpi']: fftw_components.append('mpi') if main_env['use_fftw']: FindFFTW(env, fftw_components, precision=env['real_type']) if main_env['use_cuda']: FindCuda(env) FindBoost(env, ['boost/preprocessor/seq.hpp']) # Use thrust shipped with cuda if cuda is requested thrust_var = 'CUDA_ROOT' if env['use_cuda'] else 'THRUST_ROOT' FindThrust(env, env['backend'], thrust_var) if env['build_python']: FindPybind11(env) FindExpolit(env) def subdir(env, dir): "Building a sub-directory" return env.SConscript(env.File('SConscript', dir), variant_dir=env.Dir(dir, env['build_dir']), duplicate=True) def print_build_info(env): info = ("-- Tamaas ${version}\n" + "-- SCons {} (Python {}.{})\n".format(scons_version, sys.version_info.major, sys.version_info.minor) + "-- Build type: ${build_type}\n" + "-- Thrust backend: ${backend}\n" + ("-- FFTW threads: ${fftw_threads}\n" if env['use_fftw'] else '') + "-- MPI: ${use_mpi}\n" + "-- Build directory: ${build_dir}\n" + ("-- Python version (bindings): $py_version" if env['build_python'] else '')) print(env.subst(info)) # ------------------------------------------------------------------------------ # Main compilation # ------------------------------------------------------------------------------ # Compilation colors colors = { 'cyan': '\033[96m', 'purple': '\033[95m', 'blue': '\033[94m', 'green': '\033[92m', 'yellow': '\033[93m', 'gray': '\033[38;5;8m', 'orange': '\033[38;5;208m', 'red': '\033[91m', 'end': '\033[0m' } # Inherit all environment variables (for CXX detection, etc.) main_env = Environment( ENV=os.environ, ) # Set tamaas information for k, v in TAMAAS_INFOS._asdict().items(): main_env[k] = v main_env['COLOR_DICT'] = colors main_env.AddMethod(subdir, 'SubDirectory') # Build variables vars = Variables('build-setup.conf') vars.AddVariables( EnumVariable('build_type', 'Build type', 'release', allowed_values=('release', 'profiling', 'debug'), ignorecase=2), EnumVariable('backend', 'Thrust backend', 'omp', allowed_values=('cpp', 'omp', 'tbb', 'cuda'), ignorecase=2), EnumVariable('fftw_threads', 'Threads FFTW library preference', 'omp', allowed_values=('omp', 'threads', 'none'), ignorecase=2), EnumVariable('sanitizer', 'Sanitizer type', 'none', allowed_values=('none', 'memory', 'leaks', 'address'), ignorecase=2), PathVariable('prefix', 'Prefix where to install', '/usr/local'), # Dependencies paths PathVariable('FFTW_ROOT', 'FFTW custom path', os.getenv('FFTW_ROOT', ''), PathVariable.PathAccept), PathVariable('THRUST_ROOT', 'Thrust custom path', os.getenv('THRUST_ROOT', ''), PathVariable.PathAccept), PathVariable('BOOST_ROOT', 'Boost custom path', os.getenv('BOOST_ROOT', ''), PathVariable.PathAccept), PathVariable('CUDA_ROOT', 'Cuda custom path', os.getenv('CUDA_ROOT', ''), PathVariable.PathAccept), # Dependencies provided as submodule get different default PathVariable('GTEST_ROOT', 'Googletest custom path', os.getenv('GTEST_ROOT', '#third-party/googletest/googletest'), PathVariable.PathAccept), PathVariable('PYBIND11_ROOT', 'Pybind11 custom path', os.getenv('PYBIND11_ROOT', '#third-party/pybind11/include'), PathVariable.PathAccept), PathVariable('EXPOLIT_ROOT', 'Expolit custom path', os.getenv('EXPOLIT_ROOT', '#third-party/expolit/include'), PathVariable.PathAccept), # Executables ('CXX', 'Compiler', os.getenv('CXX', 'g++')), ('MPICXX', 'MPI Compiler wrapper', os.getenv('MPICXX', 'mpicxx')), ('py_exec', 'Python executable', 'python3'), # Compiler flags ('CXXFLAGS', 'C++ compiler flags', os.getenv('CXXFLAGS', "")), # Cosmetic BoolVariable('verbose', 'Activate verbosity', False), BoolVariable('color', 'Color the non-verbose compilation output', False), # Tamaas components BoolVariable('build_tests', 'Build test suite', False), BoolVariable('build_python', 'Build python wrapper', True), # Documentation ListVariable('doc_builders', 'Generated documentation formats', default='html', names=Split("html man")), # TODO include latex # Dependencies - BoolVariable('use_googletest', 'Build tests using GTest', False), BoolVariable('use_mpi', 'Builds multi-process parallelism', False), # Distribution options BoolVariable('strip_info', 'Strip binary of added information', False), BoolVariable('build_static_lib', "Build a static libTamaas", False), # Type variables EnumVariable('real_type', 'Type for real precision variables', 'double', allowed_values=('double', 'long double')), EnumVariable('integer_type', 'Type for integer variables', 'int', allowed_values=('int', 'long')), ) # Set variables of environment vars.Update(main_env) help_text = vars.GenerateHelpText(main_env) help_text += """ Commands: scons [build] [options]... Compile Tamaas (and additional modules/tests) scons install [prefix=/your/prefix] [options]... Install Tamaas to prefix scons dev Install symlink to Tamaas python module (useful to development purposes) scons test Run tests with pytest scons doc Compile documentation with Doxygen and Sphinx+Breathe scons archive Create a gzipped archive from source """ # noqa Help(help_text) # Save all options, not just those that differ from default with open('build-setup.conf', 'w') as setup: for option in vars.options: setup.write("# " + option.help.replace('\n', '\n# ') + "\n") setup.write("{} = '{}'\n".format(option.key, main_env[option.key])) main_env['should_configure'] = \ not main_env.GetOption('clean') and not main_env.GetOption('help') build_type = main_env['build_type'] build_dir = 'build-${build_type}' main_env['build_dir'] = main_env.Dir(build_dir) # Setting up the python name with version if main_env['build_python']: args = (main_env.subst("${py_exec} -c").split() + ["from distutils.sysconfig import get_python_version;" "print(get_python_version())"]) main_env['py_version'] = bytes(check_output(args)).decode() verbose = main_env['verbose'] # Remove colors if not set if not main_env['color']: for key in colors: colors[key] = '' if not verbose: main_env['CXXCOMSTR'] = main_env['SHCXXCOMSTR'] = \ u'{0}[Compiling ($SHCXX)] {1}$SOURCE'.format(colors['green'], colors['end']) main_env['LINKCOMSTR'] = main_env['SHLINKCOMSTR'] = \ u'{0}[Linking] {1}$TARGET'.format(colors['purple'], colors['end']) main_env['ARCOMSTR'] = u'{}[Ar]{} $TARGET'.format(colors['purple'], colors['end']) main_env['RANLIBCOMSTR'] = \ u'{}[Randlib]{} $TARGET'.format(colors['purple'], colors['end']) main_env['PRINT_CMD_LINE_FUNC'] = pretty_cmd_print main_env['INSTALLSTR'] = \ u'{}[Installing] {}$SOURCE to $TARGET'.format(colors['blue'], colors['end']) # Include paths main_env.AppendUnique(CPPPATH=['#/src', '#/src/core', '#/src/surface', '#/src/percolation', '#/src/model', '#/src/model/elasto_plastic', '#/src/solvers', '#/src/gpu', '#/python']) # Changing the shared object extension main_env['SHOBJSUFFIX'] = '.o' # Variables for clarity main_env['use_cuda'] = main_env['backend'] == "cuda" main_env['use_fftw'] = not main_env['use_cuda'] main_env['use_mpi'] = main_env['use_mpi'] and not main_env['use_cuda'] if not main_env['use_fftw']: main_env['fftw_threads'] = 'none' # Back to gcc if cuda is activated if main_env['use_cuda'] and "g++" not in main_env['CXX']: raise StopError('GCC should be used when compiling with CUDA') # Printing some build infos if main_env['should_configure']: print_build_info(main_env) # OpenMP flags - compiler dependent omp_flags = { "g++": ["-fopenmp"], "clang++": ["-fopenmp"], "icpc": ["-qopenmp"] } def cxx_alias(cxx): for k in omp_flags.keys(): if k in cxx: return k raise StopError('Unsupported compiler: ' + cxx) cxx = cxx_alias(main_env['CXX']) # Setting main compilation flags main_env['CXXFLAGS'] = Split(main_env['CXXFLAGS']) main_env['LINKFLAGS'] = main_env['CXXFLAGS'] main_env.AppendUnique( CXXFLAGS=Split('-std=c++14 -Wall -Wextra'), CPPDEFINES={ 'TAMAAS_LOOP_BACKEND': 'TAMAAS_LOOP_BACKEND_${backend.upper()}', 'TAMAAS_FFTW_BACKEND': 'TAMAAS_FFTW_BACKEND_${fftw_threads.upper()}' }, ) if main_env['backend'] != 'cuda': main_env.AppendUnique(CXXFLAGS=['-pedantic']) # Adding OpenMP flags if main_env['backend'] == 'omp': main_env.AppendUnique(CXXFLAGS=omp_flags[cxx]) main_env.AppendUnique(LINKFLAGS=omp_flags[cxx]) else: main_env.AppendUnique(CXXFLAGS=['-Wno-unknown-pragmas']) # Correct bug in clang? if main_env['backend'] == 'omp' and cxx == "clang++": main_env.AppendUnique(LIBS=["atomic"]) elif main_env['backend'] == 'tbb': main_env.AppendUnique(LIBS=['tbb']) # Manage MPI compiler if main_env['use_mpi']: main_env['CXX'] = '$MPICXX' main_env.AppendUnique(CPPDEFINES=['TAMAAS_USE_MPI']) main_env.AppendUnique(CXXFLAGS=['-Wno-cast-function-type']) # Flags and options if main_env['build_type'] == 'debug': main_env.AppendUnique(CPPDEFINES=['TAMAAS_DEBUG']) # Define the scalar types main_env.AppendUnique(CPPDEFINES={'TAMAAS_REAL_TYPE': '${real_type}', 'TAMAAS_INT_TYPE': '${integer_type}'}) # Compilation flags cxxflags_dict = { "debug": Split("-g -O0"), "profiling": Split("-g -O3 -fno-omit-frame-pointer"), "release": Split("-O3") } if main_env['sanitizer'] != 'none': if main_env['backend'] == 'cuda': raise StopError( "Sanitizers with cuda are not yet supported!") cxxflags_dict[build_type].append('-fsanitize=${sanitizer}') main_env.AppendUnique(CXXFLAGS=cxxflags_dict[build_type]) main_env.AppendUnique(SHLINKFLAGS=cxxflags_dict[build_type]) main_env.AppendUnique(LINKFLAGS=cxxflags_dict[build_type]) if main_env['should_configure']: basic_checks(main_env) detect_dependencies(main_env) # Writing information file main_env.Tool('textfile') main_env['SUBST_DICT'] = get_git_subst() # Empty values if requested if main_env['strip_info']: for k in main_env['SUBST_DICT']: main_env['SUBST_DICT'][k] = "" # Substitution of environment file main_env['SUBST_DICT'].update({ '@build_type@': '$build_type', '@build_dir@': '${build_dir.abspath}', '@build_version@': '$version', '@backend@': '$backend', }) # Environment file content env_content = """export PYTHONPATH=@build_dir@/python:$$PYTHONPATH export LD_LIBRARY_PATH=@build_dir@/src:$$LD_LIBRARY_PATH """ # Writing environment file env_file = main_env.Textfile( main_env.File('tamaas_environment.sh', main_env['build_dir']), env_content) # Default targets build_targets = ['build-cpp', env_file] install_targets = ['install-lib'] if main_env._get_major_minor_revision(scons_version)[0] >= 4: main_env.Tool('compilation_db') main_env.CompilationDatabase(PRINT_CMD_LINE_FUNC=pretty_cmd_print) # Building Tamaas library Export('main_env') main_env.SubDirectory('src') # Building Tamaas extra components for dir in ['python', 'tests']: if main_env['build_{}'.format(dir)] and not main_env.GetOption('help'): main_env.SubDirectory(dir) build_targets.append('build-{}'.format(dir)) # Building API + Sphinx documentation if requested main_env.SubDirectory('doc') main_env.Alias('doc', 'build-doc') install_targets.append('install-doc') # Define dummy dev command when python is deactivated if not main_env['build_python']: dummy_command(main_env, 'dev', 'Command "dev" does not do anything' + ' without python activated ("build_python=True")') else: install_targets.append('install-python') # Define dummy test command when tests are deactivated if not main_env['build_tests']: dummy_command(main_env, 'test', 'Command "test" does not do anything' + ' without tests activated ("build_tests=True")') # Definition of target aliases, a.k.a. sub-commands main_env.Alias('build', build_targets) # Define proper install targets main_env.Alias('install', install_targets) # Default target is to build stuff main_env.Default('build') # Building a tar archive archive = main_env.Command( 'tamaas-${version}.tar.gz', '', ('git archive ' '--format=tar.gz ' '--prefix=tamaas/ ' '-o $TARGET HEAD'), ) main_env.Alias('archive', archive) diff --git a/site_scons/detect.py b/site_scons/detect.py index b089d20..6ad8ed7 100644 --- a/site_scons/detect.py +++ b/site_scons/detect.py @@ -1,260 +1,256 @@ # -*- coding: utf-8 -*- # # Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), # Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published # by the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . from __future__ import print_function from SCons.Script import Configure, Dir from SCons.Errors import StopError # ------------------------------------------------------------------------------ def _get_path(env, ext, module_var): path = "" if module_var in env and env[module_var] != "": root = Dir(env[module_var]) if not root.exists() or not root.isdir(): raise RuntimeError("{} is set to a non-existing path '{}'".format( module_var, root)) path = Dir(ext, root) if not path.exists() or not path.isdir(): raise RuntimeError("{} does not contain '{}' directory".format( module_var, ext)) return [path] # ------------------------------------------------------------------------------ def FindFFTW(env, components=None, precision='double', module_var='FFTW_ROOT'): """Find FFTW3 and set relevant environment variables""" if not env.get('should_configure', True): return if components is None: components = [] fftw_vars = {} fftw_vars['CPPPATH'] = _get_path(env, 'include', module_var) fftw_vars['LIBPATH'] = _get_path(env, 'lib', module_var) try: fftw_vars['LIBPATH'] += _get_path(env, 'lib64', module_var) except RuntimeError: pass fftw_vars['RPATH'] = fftw_vars['LIBPATH'] if 'threads' in components: fftw_vars['LIBS'] = ['pthread'] # Setting up FFTW wishes = ['main'] + components fftw_name = "fftw3{}" # Components lib_names = {'main': fftw_name, 'threads': fftw_name + '_threads', 'omp': fftw_name + '_omp', 'mpi': fftw_name + '_mpi'} # Checking list of wishes try: libs = [lib_names[i].format("") for i in wishes] except KeyError: raise StopError( 'Incompatible FFTW wishlist {0}'.format(wishes)) # Add long precision libraries if precision == 'long double': libs += [lib_names[i].format("l") for i in wishes] conf_env = env.Clone(**fftw_vars) conf = Configure(conf_env) for lib in libs: inc_names = ['fftw3.h'] if 'mpi' in lib: inc_names.append('fftw3-mpi.h') if not conf.CheckLibWithHeader(lib, inc_names, 'c++'): raise StopError( ('Failed to find library {0} or ' 'headers {1}. Check the build options "fftw_threads"' ' and "FFTW_ROOT".').format(lib, str(inc_names))) conf_env = conf.Finish() fftw_vars['LIBS'] = fftw_vars.get('LIBS', []) + libs # Update modified variables env.AppendUnique(**fftw_vars) # ------------------------------------------------------------------------------ def FindBoost(env, headers=['boost/version.hpp'], module_var='BOOST_ROOT'): """Find Boost and set relevant environment variables""" if not env.get('should_configure', True): return boost_vars = {} boost_vars['CPPPATH'] = _get_path(env, 'include', module_var) conf_env = env.Clone(**boost_vars) conf = Configure(conf_env) if not conf.CheckCXXHeader(headers): raise StopError( 'Failed to find Boost headers {}'.format(headers)) conf_env = conf.Finish() # Update modified variables env.AppendUnique(**boost_vars) # ------------------------------------------------------------------------------ def FindThrust(env, backend='omp', module_var='THRUST_ROOT'): """Find Thrust and set relevant environment variables""" if not env.get('should_configure', True): return if backend not in ('cpp', 'omp', 'cuda', 'tbb'): raise StopError( 'Unknown thrust backend "{}"'.format(backend)) thrust_vars = {} try: thrust_vars['CPPPATH'] = _get_path(env, 'include', module_var) except RuntimeError: thrust_vars['CPPPATH'] = _get_path(env, '', module_var) if "clang++" in env['CXX']: thrust_vars['CXXFLAGS'] = ["-Wno-unused-local-typedef"] thrust_vars['CPPDEFINES'] = [ "THRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_{}".format(backend.upper()) ] if backend == 'cuda': thrust_vars['CPPDEFINES'].append("TAMAAS_USE_CUDA") conf_env = env.Clone(**thrust_vars) conf = Configure(conf_env) if not conf.CheckCXXHeader('thrust/version.h'): raise StopError( 'Failed to find Thrust') conf_env = conf.Finish() # Update modified variables env.AppendUnique(**thrust_vars) # ------------------------------------------------------------------------------ def FindPybind11(env, module_var='PYBIND11_ROOT'): """Detech Pybind11 and set appropriate build variables""" if not env.get('should_configure', True) \ or env.get("PYBIND11_FOUND", False): return pybind11_vars = {} clone = env.Clone(CPPPATH=[]) clone.ParseConfig('${py_exec}-config --includes') clone.AppendUnique(CPPPATH=['$PYBIND11_ROOT']) pybind11_vars['CPPPATH'] = clone['CPPPATH'] conf = Configure(clone) if not conf.CheckCXXHeader('pybind11/pybind11.h'): raise StopError( 'Failed to find pybind11 header\n' + "Run 'git submodule update --init --recursive " + "third-party/pybind11'") conf.Finish() pybind11_vars['PYBIND11_FOUND'] = True # Update variables env.AppendUnique(**pybind11_vars) # ------------------------------------------------------------------------------ def FindCuda(env, module_var="CUDA_ROOT"): """Detect cuda on clusters""" if not env.get('should_configure', True): return if 'CUDA_ROOT' in env: env['CUDA_TOOLKIT_PATH'] = _get_path(env, '', module_var)[0] else: env['CUDA_TOOLKIT_PATH'] = '/opt/cuda' env['CUDA_COMPONENTS'] = ['cufft'] env['CUDA_ARCH_FLAG'] = '-arch=sm_60' colors = env['COLOR_DICT'] if not env['verbose']: env['SHCXX'] = 'nvcc' env['NVCCCOMSTR'] = env['SHCXXCOMSTR'] env['SHLINKCOMSTR'] = \ u'{0}[Linking (cuda)] {1}$TARGET'.format(colors['purple'], colors['end']) env.AppendUnique(CXXFLAGS=[ "-expt-extended-lambda", # experimental lambda support "-expt-relaxed-constexpr", # experimental constexpr ]) if env['build_type'] == 'debug': env.AppendUnique(CXXFLAGS="-G") env.Tool('nvcc') # ------------------------------------------------------------------------------ def FindGTest(env): """A tool to configure GoogleTest""" if not env.get('should_configure', True): return conf = Configure(env) - if not conf.CheckCXXHeader('gtest/gtest.h'): - raise StopError( - 'Failed to find GoogleTest header\n' + - "Run 'git submodule update --init --recursive " + - "third-party/googletest'") + env['has_gtest'] = conf.CheckCXXHeader('gtest/gtest.h') env = conf.Finish() # ------------------------------------------------------------------------------ def FindExpolit(env): """A tool to configure Expolit""" if not env.get('should_configure', True): return expolit_vars = {"CPPPATH": "#/third-party/expolit/include"} conf_env = env.Clone() conf_env.AppendUnique(**expolit_vars) conf = Configure(conf_env) if not conf.CheckCXXHeader('expolit/expolit'): raise StopError( 'Failed to find Expolit header\n' + "Run 'git submodule update --init " + "third-party/expolit'") conf_env = conf.Finish() env.AppendUnique(**expolit_vars) diff --git a/tests/SConscript b/tests/SConscript index 15a77a1..49b63da 100644 --- a/tests/SConscript +++ b/tests/SConscript @@ -1,216 +1,219 @@ # -*- mode:python; coding: utf-8 -*- # vim: set ft=python: # # Copyright (©) 2016-2021 EPFL (École Polytechnique Fédérale de Lausanne), # Laboratory (LSMS - Laboratoire de Simulation en Mécanique des Solides) # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published # by the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . from __future__ import print_function from SCons.Script import Split, Copy, Dir, Import from detect import FindGTest, FindPybind11 # ------------------------------------------------------------------------------ def copyComStr(env, main): if 'SHCXXCOMSTR' in main: env['CXXCOMSTR'] = main['SHCXXCOMSTR'] if 'SHLINKCOMSTR' in main: env['LINKCOMSTR'] = main['SHLINKCOMSTR'] # ------------------------------------------------------------------------------ def make_python_tests(env): """Copy python tests to build directory""" test_env = env.Clone() test_files = Split(""" test_hertz.py test_westergaard.py test_patch_westergaard.py test_patch_plasticity.py test_surface.py test_hertz_disp.py test_hertz_kato.py test_saturated_pressure.py test_flood_fill.py test_integral_operators.py test_dumper.py test_tangential.py test_boussinesq_surface.py test_voigt.py test_memory.py test_epic.py fftfreq.py conftest.py pytest.ini """) if env['use_mpi']: test_files += ['test_mpi_routines.py', 'mpi_routines.py'] src_dir = "#/tests" targets = [ test_env.Command(file, test_env.File(file, src_dir), Copy("$TARGET", "$SOURCE")) for file in test_files ] test_env = env.Clone(tools=[pybind11]) # Helper module for integral operators test_env['SHLIBPREFIX'] = '' test_env.PrependUnique(LIBS=['Tamaas']) register = test_env.Pybind11Module( target="register_integral_operators", source=["register_integral_operators.cpp"]) Import('libTamaas') test_env.Depends(register, libTamaas) targets.append(register) return targets # ------------------------------------------------------------------------------ def compile_google_test(env, gtest_path): gtest_obj = env.Object('gtest.o', [env.File("src/gtest-all.cc", gtest_path)]) return env.StaticLibrary('gtest', gtest_obj) # ------------------------------------------------------------------------------ def make_google_tests(env): gtest_dir = Dir(env['GTEST_ROOT']) gtest_env = env.Clone(CPPPATH=[gtest_dir], CXXFLAGS=['-pthread', '-isystem', env.Dir('include', gtest_dir).path]) FindGTest(gtest_env) + if not gtest_env['has_gtest']: + return [] + libgtest = None # Hugly hack to detect if we need to compile gtest submodule if env['GTEST_ROOT'] == '#third-party/googletest/googletest': gtest_path = str(gtest_dir) libgtest = compile_google_test(gtest_env, gtest_path) env.AppendUnique(CXXFLAGS=gtest_env['CXXFLAGS']) env.PrependUnique(LIBS=['Tamaas', env.subst('python${py_version}')]) google_test_files = Split(""" test_grid.cpp test_loop.cpp test_model.cpp test_static_types.cpp test_integration.cpp """) if env['use_fftw']: google_test_files += ['test_fftw.cpp'] if env['use_cuda']: google_test_files += ['test_cufft.cpp'] # Necessary for the tests that use pybind11 calls to python uses = [] if env['build_python']: google_test_files.append('test_fftfreq.cpp') uses = ['TAMAAS_USE_PYTHON'] if env['use_mpi']: google_test_files.append('test_mpi.cpp') defines = env['CPPDEFINES'] if type(defines) is not list: defines = [defines] gtest_main = env.Object("tamaas_gtest_main.o", 'tamaas_gtest_main.cc', CPPDEFINES=defines + uses) gtest_all = env.Program('test_gtest_all', google_test_files + [gtest_main], LIBS=(env['LIBS'] + ['gtest'])) Import('libTamaas') env.Depends(gtest_all, libTamaas) env.Depends(gtest_all, libgtest) - return [gtest_all] + copy_gtest = gtest_env.Command('test_gtest.py', '#tests/test_gtest.py', + Copy('$TARGET', '$SOURCE')) + + return [gtest_all, copy_gtest] # ------------------------------------------------------------------------------ def make_bare_tests(env): rough = env.Program("test_rough_surface.cpp") Import('libTamaas') env.Depends(rough, libTamaas) return [rough] # ------------------------------------------------------------------------------ Import('main_env') # Setup of test environment test_env = main_env.Clone() test_env.AppendUnique( LIBPATH=['.', '../src'], RPATH=["'$$$$ORIGIN/../src'"] ) test_env.PrependUnique(LIBS=['Tamaas']) # Building tests that do not require any third party targets = make_bare_tests(test_env) # Build tests that required python bindings if test_env['build_python']: FindPybind11(test_env) test_env.Tool(pybind11) test_env.ParseConfig("${py_exec}-config --ldflags") test_env['CCFLAGS'] = [] targets += make_python_tests(test_env) # Building google tests -if test_env['use_googletest']: - targets += make_google_tests(test_env) - targets.append(test_env.Command('test_gtest.py', '#tests/test_gtest.py', - Copy('$TARGET', '$SOURCE'))) +targets += make_google_tests(test_env) # Target alias to build tests main_env.Alias('build-tests', targets) # Check if pytest is installed conf = Configure(test_env, custom_tests={'CheckPythonModule': CheckPythonModule}) has_pytest = conf.CheckPythonModule('pytest') conf.Finish() # Define a command to execute tests if has_pytest: pytest_env = test_env.Clone() test_env['pythonpath'] = '${build_dir}/python' test_env['ld_library_path'] = '${build_dir}/src' pytest_env.PrependENVPath('PYTHONPATH', '${pythonpath.abspath}') pytest_env.PrependENVPath('LD_LIBRARY_PATH', '${ld_library_path.abspath}') # Setting a moderate thread number pytest_env['ENV']['OMP_NUM_THREADS'] = "1" pytest_env['PYTESTOPTS'] = ['${"-v" if verbose else "-q"}'] test_target = pytest_env.Command( '.phony_test', targets, '${py_exec} -m pytest $PYTESTOPTS ${TARGET.dir}') main_env.Alias('test', test_target) else: # We still define a target here so that `scons test` still works dummy_command(main_env, 'test', 'Cannot run tests: pytest is not installed')