+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Scientific Programming
for Engineers (SP4E)
+for Engineers (SP4E)
+
+
+
+ Welcome ¶
-
+
- About the class
-
+
- The goal is to provide some useful knowledge (tool box) to the job of a PhD student who wants to perform a numerical work +
- Uncover the details of computers and numerical simulations: bring power (analogy blue/red pill of Matrix movie) +
- The class will be a bit particular (uncovering knowledge), thus it is personal and needs participation. +
- Students have to build their own understanding/experience on the various tasks required. +
- Evaluation will be made by evaluating homeworks. +
+
+
+
+
+Difficulties in making scientific software¶
+
+
+
+
+-
+
- Real numbers (and all rounding errors)
-
+
- overflow and catastrophic cancellation (evaluation of velocity) +
- Famous disaster about patriot missiles +
+ - Units
-
+
- Conversion of units to be able to plug modules together +
- Non SI unit system needs explicit conversion between quantities +
- Mars Orbiter disaster +
+ Rapid changes
+-
+
- Need of quick and easy implementation/insertion of new features +
- Need non regression tests +
+Large problems
+-
+
- Need to manipulate large number of DOFs to increase accuracy +
- Need for collaborative tools to make cutting edge applications +
+
+
+
+
+
+ "*computer scientists are/should be LAZY (but smartly)*" ¶
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Cycles of engineering production of software ¶
+
+
+
+
+-
+
- Scientific question +
- Analysis and formulation of the problem. +
- Conception: algorithmic decision. +
- Implementation +
- Deployment of the code +
- Analysis/post-treatment of the results +
+
+
+
+
+ Scientific question ¶
+
+
+
+
+ Analysis and formulation of the problem ¶
-
+
- Math formulation: PDE, optimization, statistical, ... +
- Define the input and output of the model +
- Separate the post-treatment +
- If there are some analytical solutions you need to be aware +
+
+
+
+ Conception: algorithmic decision ¶
-
+
- Make a chart diagram of the algorithm: separate the subparts +
- Choose/identify how to solve the subparts +
- Choose the appropriate data structures +
- For new algorithms, evaluate the complexity, make an effort of generalization +
- If global complexity unaffordable with the desired grain (number of DOFs) consider changing the model (back to 2:formulation ). +
+
+
+
+ Implementation ¶
-
+
- Identify existing software for the entire project and for the subparts +
- Decide a programming language +
- Decide a coding convention (question of style) +
- Decide where the code is hosted (for backups and revisions) +
- Decide a source documentation format +
- Program the thing +
- Setup of tests for each necessary sub-part (during programming) +
- Debug and check for memory leaks +
- The programming feasibility could here lead to change algorithms or model (back to 2:formulation or 3:conception) +
+
+
+
+ Deployment of the code ¶
-
+
- Identify the target machine +
- Fix compilation issues (multi-system, portability, architecture CPU vs. GPU) +
- Parametric study management (scripting the launch) +
- Computing shared/cluster of resources: queue system management +
- Compilation issue, version issues, could lead to implementation change (back to 4:implementation) +
- Specific hardware could invalidate the algorithmic decision (back to 3:conception) +
+
+
+
+ Analysis/post-treatment of the results ¶
-
+
- Sorting/treating the output +
- Compute digested measures for the brute output (this should be faster than the computation itself otherwise back to 2:formulation) +
- Produce graphs (for papers and presentations) +
- 3D vizualization +
- Validate numerical results, stability, coherence, comparison to experiment +
- If the validation fail get back to 5,4,3 or 2 is possible +
+
+
+
+ The method for this class ¶
-
+
- Computer place: Your laptop
-
+
- Linux as main OS, Dualboot, or on a VirtualBox(might get slow) +
- Python framework +
- C++ framework +
+
+
+
+
+-
+
- Open source philosophy $\Rightarrow$ access to source
-
+
- Possibility to check the content of the code +
- Possibility to modify the code for your own purpose +
- Power of the community +
- Business model based on knowledge rather than license +
+
+
+
+
+ The method for this class ¶
-
+
- Why C++ and in python
-
+
- Object oriented is a way to organize the programs for re-usability +
- C++ allows objects and efficiency +
- Python allows interactivity and quick testing +
+
+
+
+
+-
+
- On the repositories
-
+
- Program = bug +
- Since programs can be complex we need a tool to tag source revisions +
- Collaborative tool to software development +
- cvs, svn, git. There is a git at EPFL that we will use. +
+
+
+
+
+ The method for this class ¶
-
+
- On post-treatment programs
-
+
- Python is perfect for manipulating files, for parsing outputs, for making graphs +
- Many program exists for visualization. An opensource one: Paraview +
+
+
+
+
+
+ Class plan description ¶
+
+
+
+
+
+
+
+
+
+
+
+
+-
+
- Tuesday 18-th Sep: Introduction, GIT, python hello world +
- Tuesday 25-th Sep: C++ hello world, floating point numbers, pi computation +
- Tuesday 02-th Oct: C++ STL, Coding convention (C++11) +
- Tuesday 09-th Oct: Paraview, Numpy&Scipy, Matplotlib, Exercise: conjugate gradient (Homework) +
+
+
+
+
+
+
+
+
+
+
+
+-
+
- Tuesday 16-th Oct: Object oriented with python +
- Tuesday 23-th Oct: Object oriented with C++ (Homework) +
- Tuesday 30-th Oct: Interactive session to design Particle's code, Design patterns, Implementation of simple kepler orbit +
- Tuesday 06-th Nov: Continuing adimentional kepler's orbit +
+
+
+
+
+ Class plan description ¶
+
+
+
+
+
+
+
+
+
+
+
+
+-
+
- Tuesday 13-th Nov: Google test. Exercise Produce a test suite for particle's code +
- Tuesday 20-th Nov: Code optimization and Templates + pingpong ball. (Homework) +
- Tuesday 27-th Nov: Homework about trajectory optimization (Homework) +
+
+
+
+
+
+
+
+
+
+
+
+-
+
- Tuesday 04-th Dec: Using external libraries. Exercise with FFTW (porting to C++) +
- Tuesday 11-th Dec: Template library: Eigen. Exercise: mass-spring equilibrium +
- Tuesday 18-th Dec: PyBind+cppimport. Exercise porting the particle's code to python +
+
+
+
+
+
+
+ The GIT ¶
+
+
+
+What would you demand to a tool that will hold your program sources ?¶
+
+
+
+
+-
+
- Manage history (evolution in time) +
- Rewind time +
- Transport/Backup through network +
- Team/Concurrent working +
This is the standard of most Version control systems such as GIT
+ +
+
+
+
+
+ Revision manager:
The GIT & c4science
+The GIT & c4science
-
+
- Git is a free distributed version control system (DVCS), used for source code management (SCM) +
- Git operates on a decentralized architecture, so every git working directory has the complete history +
- Git was initiallydesigned and created by Linus Torvalds for Linux kernel development +
- EPFL has a GIT repository service (http://c4science.ch/) +
+
+
+
+
+
+
+-
+
- The working copy is the state (can be modified) of a selected branch (definition comes later) +
- To know the status of the working copy:
+
+git status
+ - See the log
+
+git log
+
+
+
+
+
+
+
+GIT - Branches¶
-
+
- All modern VCSs have a mechanism for branches +
- Branching means you diverge from the main line of development and continue without perturbing the code +
- Branches can evolve independently +
- The main branch in GIT is usually called master +
- GIT doc on branches +
+
+
+
+-
+
- See/create branches:
+
+git branch
+ - Change the working copy to another branch
+
+git checkout stable-branch
+
+
+
+
+GIT - Push your modifications¶
+git push origin master-
+
- This operation sends the current branch and merge it into the remote branch +
+
+
+
+GIT - Pull modifications¶
+git pull origin master-
+
- This operation actually fetch the remote branch and merge into current branch +
+
+
+
+GIT - remotes¶
-
+
- You can pull/push to more than a single remote +
- list the declared remotes:
+
+git remote -v
+ - add/remove remotes
+
+git remote add/remove
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+GIT - commands¶
git log
+git status
+git checkout
+git add file.cc
+git rm file.cc
+git mv file.cc
+git commit -m "nice message" file.cc
+git push remote_name branch_name
+git push origin master
+git diff
+git diff revision_hash
+git help whatever_command
+
+
+
+
+
+
+ Implicit rule: do not commit something
that is not compiling
to a shared/public repository
+
+to a shared/public repository
+
+
+
+GIT - resources¶
-
+
- Cheat Sheet: http://ndpsoftware.com/git-cheatsheet.html +
- Simple guide: http://rogerdudler.github.io/git-guide/ +
- Nice tutorial: http://learngitbranching.js.org/ +
+
+
+
+c4science.ch¶
What is c4science ?
+C4 Science is a co-creation platform, curation and code sharing. This platform includes:
+-
+
- Version management system +
- Common authentication to all Swiss universities to local + external collaborators +
- Social dimension (wikis, bug tracking, ...) +
- Code test system (continuous integration) +
- Swiss alternative to github +
+
+
+
+c4science.ch¶
Connect to c4science
+The recommended way to connect to the c4science server (and actually any distant linux machine) is through the SSH protocol:
+-
+
- You need a pair of keys: one public and one private +
- They are stored in the directory .ssh in your home directory +
- The public can be distributed, the private should stay secret +
- A good habit is to generate one key-pair per client and never transport the private key +
+
+
+
+Interactive session on manipulating git¶
-
+
- installing git +
- creating a repository on c4science +
- cloning it to private laptop +
- fix permissions +
- manage conflicts +
+
+
+
+
+
+Python language
concepts and syntax
+
+concepts and syntax
+
+
+ Please install Python/Anaconda on you laptop
+
+
+
+
+
+
+
+
+
+
+-
+
- comments +
#my super comment
+-
+
- command print +
print(10)
+-
+
- type +
a = 10
+b = 10.
+type(a)
+type(b)
+-
+
- Converting string to integer or float +
a = "1.2"
+print(a)
+a_int = int(a)
+print(a_int)
+a_float = float(a)
+print(a_float)
+
+
+
+
+-
+
- list+
a = list() +a.append(2) +print(a) +a = [2,4,5] +a.append([2,4,5]) +print(a) +a += [2,4,5] +b = [1,2]*4 +print(a[1]) +print(a[2]) +
+ - maps/dictionaries+
planet = dict() +planet["name"] = "mars" +planet["radius"] = 1.2 +planet["mass"] = 0.4 +print(planet) +
+
+
+
+
+-
+
- blocks in python identified by ':' and with the indentation+
block_start: + #start of the block + instruction1 + instruction2 + instruction3 + #end of the block +
+
+
+
+
+-
+
- if conditions+
a = 1 +if not a == 0: + print("a != 0") +else: + print("a == 0") +
+
+
+
+
+
+
+
+-
+
- for loops (beware the indentation) +
for i in range(0,10):
+ print(i)
+for i in mylist:
+ print(i)
+
+
+
+
+-
+
- defining function+
def foo(arg1, arg2, arg3): + ... + some_code + ... +
+
+
+
+
+
+
+
+-
+
- opening files for writing +
f = open("my_super_filename.csv", 'w')
+f.write("#X Y Z VX VY VZ FX FY FZ mass radius name\n")
+-
+
- opening a file and read line by line +
f = open("my_super_filename.csv", 'r')
+for line in f:
+ entries = line.split()
+ print(entries)
+
+
+
+
+
+
+
+-
+
- using other modules +
# request to use another module
+import sys
+-
+
- splitting your program in several files +
#my_module.py
+
+def foo(arg):
+ return arg+1
+I can call this function from another file with
+import my_module
+
+res = my_module.foo(10)
+print(res)
+-
+
- The file must be accessible +
- To control what is accessible, the environment variable PYTHONPATH is checked +
+
+
+
+
+
+-
+
- launching a python script from the terminal +
python3 exe.py arg1 arg2 arg3-
+
- how to get the arguments passed to the program ? +
import sys
+
+# the arguments are stored in the list sys.argv
+# for instance you can print it with
+
+print(sys.argv)
+
+
+
+
+Interactive session on the Hello world¶
-
+
- Use GIT to access the first exercise +
- How to split a source file into modules +
- Manipulating program arguments (argv) +
- Series calculation +
- Debugger PDB +