diff --git a/README.md b/README.md index 17d5084..3360c18 100644 --- a/README.md +++ b/README.md @@ -1,590 +1,587 @@ # BlackDynamite
# Installation ## Dependencies ```bash sudo apt-get install python-psycopg2 sudo apt-get install python-numpy sudo apt-get install python-argcomplete ``` ## Installation of client side The easiest is through pip, that needs first to be installed: ```bash sudo apt-get install python-pip ``` Then for a system wide installation (recommended): ```bash sudo pip install git+https://c4science.ch/diffusion/3127/blackdynamite.git ``` or ```bash sudo pip install git+ssh://git@c4science.ch/diffusion/3127/blackdynamite.git ``` Then for a user scope installation: ```bash pip install --user git+https://c4science.ch/diffusion/3127/blackdynamite.git ``` or ```bash pip install --user git+ssh://git@c4science.ch/diffusion/3127/blackdynamite.git ``` ## Getting the sources You can clone the GIT repository: ```bash git clone ssh://git@c4science.ch/diffusion/3127/blackdynamite.git ``` or ```bash git clone https://c4science.ch/diffusion/3127/blackdynamite.git ``` ## Installing completion To benefit the autocompletion for **BlackDynamite** the following steps are needed. You first need to install the argcomplete modules. Either by typing (Depending of your Ubuntu/Debian version) : ```bash sudo apt-get install python-argcomplete ``` or: ```bash sudo apt-get install python-pip sudo pip install argcomplete ``` Then you must insert the following in your .bashrc ```bash eval "$(register-python-argcomplete getRunInfo.py)" eval "$(register-python-argcomplete launchRuns.py)" eval "$(register-python-argcomplete canYouDigIt.py)" eval "$(register-python-argcomplete cleanRuns.py)" eval "$(register-python-argcomplete updateRuns.py)" eval "$(register-python-argcomplete enterRun.py)" eval "$(register-python-argcomplete enterRun.py)" eval "$(register-python-argcomplete saveBDStudy.py)" ``` ## Register hosts to BlackDynamite In the .blackdynamite folder (in your home) you should add the servers where your databases are, with the option and information of your choice. For each database you can add a file .bd of the name of the server (or an alias and specify the host inside: ```bash host = yourHost.domain.countryID ``` It is also recommended to specify the password of the database to avoid typing it when using auto-completion. Here is an example of a valid blackdynamite config file: ```bash cat ~/.blackdynamite/lsmssrv1.epfl.ch.bd ``` ```bash host = lsmssrv1.epfl.ch password = XXXXXXXXX ``` # Introduction and philosophy **Blackdynamite** is merely a tool to help achieving a few things: 1) Launching a program repeatedly with varying parameters, to explore the chosen parametric space. 2) Collect and sort results of **Small sizes** benefiting from the power of modern databases. 3) Analyze the results by making requests to the associated databases. **Launching** is made simple by allowing any executable to be launched. The set of directories will be generated and managed by BlackDynamite to prevent errors. Requests of any kind will then be made to the underlying database through friendly commands of BlackDynamite. **Collecting** the results will be possible thanks to the Blackdynamite C/C++ and python API which will let you send results directly to the database and thus automatically sort them. This is extremely useful. However heavy data such as Paraview files or any other kind of data should not be pushed to the database for obvious performance issues. **Analysis** of the results can be made easy thanks to Blackdynamite which can retrieve data information in the form of Numpy array to be used, analyzed or plotted thanks to the powerful and vast Python libraries such as Matplotlib and Scipy. The construction of a **BlackDynamite** parametric study follows these steps: - Describing the parametric space - Creating jobs (specific points in the parametric space) - Creating runs (instances of the jobs) - Launching runs - Intrumenting the simulation to send results - Analyzing the results # Setting up a parametric study ## Chose the parameters of the study The first thing to do is to setup the table in the database associated with the study we want to perform. For this to be done you need, first of all, to list all the parameters that decide a specific case/computation. This parameters can be of simple types like string, integers, floats, etc. At current time no vectorial quantity can be considered as an input parameter. Once this list is done you need to create a script, usually named 'createDB.py' that will do this task. Let us examine such an example script. ### Setting up blackdynamite python modules First we need to set the python headers and to import the **BlackDynamite** modules by ```python #!/usr/bin/env python import BlackDynamite as BD ``` Then you have to create a generic black dynamite parser and parse the system (including the connection parameters and credentials) ```python parser = BD.BDParser() params = parser.parseBDParameters() ``` This mechanism allows to easily inherit from the parser mechanism of BlackDynamite, including the completion (if activated: see installation instructions). Then you can connect to the black dynamite database ```python base = BD.base.Base(**params) ``` ## Setting up of the parametric space: the jobs pattern Then you have to define the parametric space (at present time, the parametric space cannot be changed once the study started: be careful with your choices). Any particular job is defined as a point in the parametric space. For instance, to create a job description and add the parameters with int, float or list parameters, you can use the following python sequence. ```python myjob_desc = BD.job.Job(base) myjob_desc.types["param1"] = int myjob_desc.types["param2"] = float myjob_desc.types["param3"] = str ``` **Important remark: Do not name your parameters like PostGreSQL keywords.** ## Setting up of the run space Aside of the jobs, a run will represent a particular realisation (computation) of a job. To get clearer, the run will contain information of the machine it was run on, the executable version, or the number of processors employed. For instance creating the run pattern can be done with: ```python myruns_desc = run.Run(base) myruns_desc.types["compiler"] = str ``` There are default entries to the description of runs. These are: - machine_name: the name of the machine where the run must be executed - job_id (integer): the ID of the running job - has_started (bool): flag to know whether the job has already started - has_finished (bool): flag to know whether the job has already finished - run_name (string): the name of the run - wait_id (int): The id of a run to wait before starting - start_time (TIMESTAMP): The start time for the run ## Commit the changes to the database Then you have to request for the creation of the database ```python base.createBase(myjob_desc,myruns_desc,**params) ``` You have to launch the script. As mentioned, all BlackDynamite scripts inherit from the parsing system. So that when needing to launch one of these codes, you can always claim for the valid keywords: ```bash ./createDB.py --help usage: createDB.py [-h] [--job_constraints JOB_CONSTRAINTS] [--study STUDY] [--port PORT] [--host HOST] [--user USER] [--truerun] [--run_constraints RUN_CONSTRAINTS] [--yes] [--password] [--list_parameters] [--BDconf BDCONF] [--binary_operator BINARY_OPERATOR] BlackDynamite option parser optional arguments: -h, --help show this help message and exit General: --yes Answer all questions to yes. (default: False) --binary_operator BINARY_OPERATOR Set the default binary operator to make requests to database (default: and) BDParser: --job_constraints JOB_CONSTRAINTS This allows to constraint run selections by job properties (default: None) --study STUDY Specify the study from the BlackDynamite database. This refers to the schemas in PostgreSQL language (default: None) --port PORT Specify data base server port (default: None) --host HOST Specify data base server address (default: None) --user USER Specify user name to connect to data base server (default: None) --truerun Set this flag if you want to truly perform the action on base. If not set all action are mainly dryrun (default: False) --run_constraints RUN_CONSTRAINTS This allows to constraint run selections by run properties (default: None) --password Flag to request prompt for typing password (default: False) --list_parameters Request to list the possible job/run parameters (default: False) --BDconf BDCONF Path to a BlackDynamite file (*.bd) configuring current optons (default: None) ``` An important point is that most of the actions are only applied when the 'truerun' flag is set. Also, you always have to mention the host and the study you are working on (all scripts can apply to several studies). To launch the script and create the database you should launch: ```bash ./createDB.py --host lsmssrv1.epfl.ch --study MysuperCoolStudy --truerun ``` ## Creating the jobs The goal of the parametric study is to explore a subpart of the parametric space. We need to create jobs that are the points to explore. This script is usually named 'createJobs.py'. We need to write a python script to generate this set of jobs. We start by setting the modules and the parser as for the 'createDB.py' script. Then we need to create job object: ```bash job = job.Job(base) ``` It is up to us to decide the values to explore. for convenience, it is possible to insert ranges of values: ```bash job["param1"] = 10 job["param2"] = [3.14,1.,2.] job["param3"] = 'toto' ``` This will create 3 jobs since we provided a range of values for the second parameter. The actual creation is made by calling: ```python base.createParameterSpace(job) ``` Launching the script is made with: ```python ./createJobs.py --host lsmssrv1.epfl.ch --study test --truerun ``` ## Creating the runs and launching them At this point the jobs are in the database. You need to create runs that will precise the conditions of the realization of the jobs. For example the machine onto which the job will run, path dependent information, executable information and others. We have to write the last script, usually named 'createRuns.py' to specify run creations. Again we start with the modules. However this time, we can use another parser class more adapted to the manipulation of runs: ```python parser = BD.RunParser() params = parser.parseBDParameters() base = BD.Base(**params) ``` The default parameters for runs will then be automatically included in the parameters. ```python myrun = run.Run(base) ``` Some of the standard parameters might have been parsed directly by the RunParser, so that we have to forward them to the Run object: ```python myrun.setEntries(params) ``` A run now specify what action to perform to realize the job. Usually, an end-user has a script(s) and wish to attach it to the run. To attach a file you can for instance do: ```python myrun.addConfigFiles(['file1','file2','launch.sh']) ``` Then, one has to specify which of these files is the entry point: ```python myrun.setExecFile("launch.sh") ``` Finally, we have to create Run objects and attach them to jobs. The very first task is to claim the jobs from the database. To that end the object JobSelector shall be your friend: ```python jobSelector = BD.JobSelector(base) job_list = jobSelector.selectJobs() ``` This will return a job list that you can loop through and attach the runs to: ```python for j in job_list: myrun['compiler'] = 'gcc' myrun.attachToJob(j) ``` Everything should then be committed to the database: ```python if params["truerun"] is True: base.commit() ``` To create the run one should eventually launch the script by typing: ```bash ./createRuns.py --host lsmssrv1.epfl.ch --study test --machine_name lsmspc41 --run_name toto --nproc int --truerun ``` The runs are eventually launched using the tool 'launchRuns.py'. ```bash ./launchRuns.py --host lsmssrv1.epfl.ch --study test --outpath /home/user/ --truerun (--nruns int) ``` ## Accessing and manipulating the database The runs can actually be controlled in the database with the tool 'getRunInfo.py', and one can go to the run folder with 'enterRun.py'. The runs are then launched using the tool 'launchRuns.py'. ```bash ./getRunInfo.py --host lsmssrv1.epfl.ch --study test ./enterRun.py --host lsmssrv1.epfl.ch --study test --run_id ID ``` The status of the run can be manually modified using the command 'cleanRuns.py', the default status is CREATED (it can be turned to delete) ```bash ./cleanRuns.py --host lsmssrv1.epfl.ch --study test (--runid ID) --truerun (--delete) ``` The status and the other run parameters (e.g. the compiler in the example file) can also be modified with 'updateRuns.py'. This can be done in the executed scrip to automatically set the selected parameter ```bash updateRuns.py --host lsmssrv1.epfl.ch --study test --updates 'state = toto' --truerun ``` The function 'canYouDigIt.py' is an example of how to collect data in the runs to draw graphs. Example to plot the crack length in function of the time for different sigma_c (the study parameter): ```bash canYouDigIt.py --host lsmssrv1.epfl.ch --study test --quantity time, crack_length --using %0.x:%1.y --xlabel 'time' --ylabel 'crack_length' --legend 'sigma_c = %j.sigma_c' ``` Eventually, the database can be saved in .zip format to be exported and used offline with 'saveBDStudy.py'. # Instrumenting a *C++* simulation code Within you program you need a pusher correctly initialized in order to push data to the database. The 'test_blackdynamite.cc' is an example of such pusher. First *blackdynamite* includes are required: ```cpp #include "blackdynamite.hh" ``` Then you need to create a Pusher object and initialize it. ```cpp BlackDynamite::RunManager bd; bd.startRun(); ``` The constructor by default reads environment variables to get the database connection and schema informations: - RUN_ID: the identifier of the runid - SCHEMA: the schema where the parametric study is to be stored - HOST: the database hostname Then in the places where values are created you push the values to the database ```cpp bd.push(val1,"quantity1",step); bd.push(val2,"quantity2",step); ``` Step is a stage identifier. It can be the step index within an explicit loop, or a within a convergence descent or whatever you whish. It will serve later to compare quantity entries. Finally, when the job ended the following call inform the database that the run is finished: ```cpp bd.endRun(); ``` # Instrumenting a *Python* simulation code Within your program you need a run object to push data to the database. This is done by selecting the run from the 'run_id' (usually passed as parameter). ```python parser = BD.RunParser() params = parser.parseBDParameters() # params['run_id'] should exist mybase = BD.Base(**params) runSelector = BD.RunSelector(mybase) myrun = runSelector(params) ``` In order to have time entries for run times, the 'start()' and 'finish()' of the run need to be called. ```python myrun.start() # ... # Important stuff # ... myrun.finish() ``` Pushing data is can be done with 'pushVectorQuantity()' and 'pushScalarQuantity()'. ```python myrun.pushVectorQuantity(vector_quantity, step, "quantity_id", is_integer=False) myrun.pushScalarQuantity(scalar_quantity, step, "quantity_id", is_integer=False) ``` # Fecthing the results Under construction... ## Installation of the server side: setting up the PostGreSQL database (for admins) If you want to setup a PostGreSQL server to store BlackDynamite data, then you have to follow this procedure. Install the PSQL server: ```bash sudo apt-get install postgresql-9.4 ``` You know need privileges to create databases and users. This can be done using the following: +You should add a database named blackdynamite (only the first time): ```bash -bash:> sudo su postgres -bash:> psql - -postgres=# +psql --command "CREATE USER blackdynamite WITH PASSWORD '';" +createdb -O blackdynamite blackdynamite ``` -Then you should create a user: -```psql -postgres=# create user mylogin; -``` +## Adding a user -configure plpgsql language to the database: -```psql -postgres=# CREATE PROCEDURAL LANGUAGE plpgsql; +You should create a user: +```bash +psql --command "CREATE USER mylogin WITH PASSWORD 'XXXXX';" ``` -And a database associated with that user: -```psql -postgres=# create database mylogin; -postgres=# alter database mylogin owner to mylogin; -postgres=# grant create on database mylogin TO mylogin; -postgres=# alter role mylogin with password 'my_pass'; +And add permissions to create tables to the user +```bash +psql --command "grant create on database blackdynamite to mylogin" ``` +This can also be done with the commodity tool +```bash +createUser.py --user admin_user --host hostname +``` # Useful Postgresql commands How to list the available schemas ? ```psql > \dn ``` How to get into the schema or the study ? ```psql > set search path to schema_name; > set search path to study_name; ``` How to list all the tables ? ```psql > \d ``` or ```psql > SELECT * FROM pg_catalog.pg_tables; ``` How to list entries from a table (like the jobs) ? ```psql > SELECT * from table_name ; ``` How to list all the databases ? ```psql > \l ``` How to list the available databases ? ```psql > select datname from pg_catalog.pg_database; ``` How to know the current database ? ```psql > select current_database(); ``` \ No newline at end of file diff --git a/doc/Black-Dynamite.png b/doc/Black-Dynamite.png deleted file mode 100644 index 505534b..0000000 Binary files a/doc/Black-Dynamite.png and /dev/null differ diff --git a/doc/manual-macros.sty b/doc/manual-macros.sty deleted file mode 100644 index d118541..0000000 --- a/doc/manual-macros.sty +++ /dev/null @@ -1,30 +0,0 @@ -% Some new commands -\newcommand{\akantu}{{\texttt{\textbf{Akantu}}}\xspace} - -\newcommand{\examplesdir}{\texttt{examples}} - -\newcommand{\code}[1]{\texttt{#1}} -\newcommand{\note}[2][]{\textbf{Note#1: }\textit{#2}} -\newcommand{\todo}[1]{~({\small\color{red}\textbf{TODO: }\textbf{#1}})} - -\renewcommand{\vec}[1]{\ensuremath{\boldsymbol{#1}}} -\newcommand{\mat}[1]{\ensuremath{\boldsymbol{#1}}} -\newcommand{\st}[1]{{\mathrm{#1}}} - -\newcommand{\eg}{\emph{e.g.},\xspace} -\newcommand{\ie}{\emph{i.e.},\xspace} - -\newcommand{\realnum}{\mathbb{R}} % -\newcommand{\intnum}{\mathbb{Z}} % Requires: -\newcommand{\natnum}{\mathbb{N}} % \usepackage{pxfonts} -\newcommand{\compnum}{\mathbb{C}} % -\newcommand{\ratnum}{\mathbb{Q}} % - -\newcommand{\half}{\ensuremath{\tfrac{1}{2}}\xspace} -\newcommand{\quart}{\ensuremath{\tfrac{1}{4}}\xspace} -\newcommand{\third}{\ensuremath{\tfrac{1}{3}}\xspace} -\newcommand{\twothird}{\ensuremath{\tfrac{2}{3}}\xspace} -\newcommand{\sixth}{\ensuremath{\tfrac{1}{6}}\xspace} -\newcommand{\eighth}{\ensuremath{\tfrac{1}{8}}\xspace} -\newcommand{\invsqrtthree}{\ensuremath{\tfrac{1}{\sqrt{3}}}\xspace} - diff --git a/doc/manual.cls b/doc/manual.cls deleted file mode 100644 index 7b94545..0000000 --- a/doc/manual.cls +++ /dev/null @@ -1,754 +0,0 @@ -%% -%% This is file `book.cls', -%% generated with the docstrip utility. -%% -%% The original source files were: -%% -%% classes.dtx (with options: `book') -%% -%% This is a generated file. -%% -%% Copyright 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 -%% The LaTeX3 Project and any individual authors listed elsewhere -%% in this file. -%% -%% This file was generated from file(s) of the LaTeX base system. -%% -------------------------------------------------------------- -%% -%% It may be distributed and/or modified under the -%% conditions of the LaTeX Project Public License, either version 1.3c -%% of this license or (at your option) any later version. -%% The latest version of this license is in -%% http://www.latex-project.org/lppl.txt -%% and version 1.3c or later is part of all distributions of LaTeX -%% version 2005/12/01 or later. -%% -%% This file has the LPPL maintenance status "maintained". -%% -%% This file may only be distributed together with a copy of the LaTeX -%% base system. You may however distribute the LaTeX base system without -%% such generated files. -%% -%% The list of all files belonging to the LaTeX base distribution is -%% given in the file `manifest.txt'. See also `legal.txt' for additional -%% information. -%% -%% The list of derived (unpacked) files belonging to the distribution -%% and covered by LPPL is defined by the unpacking scripts (with -%% extension .ins) which are part of the distribution. -%% \CharacterTable -%% {Upper-case \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z -%% Lower-case \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z -%% Digits \0\1\2\3\4\5\6\7\8\9 -%% Exclamation \! Double quote \" Hash (number) \# -%% Dollar \$ Percent \% Ampersand \& -%% Acute accent \' Left paren \( Right paren \) -%% Asterisk \* Plus \+ Comma \, -%% Minus \- Point \. Solidus \/ -%% Colon \: Semicolon \; Less than \< -%% Equals \= Greater than \> Question mark \? -%% Commercial at \@ Left bracket \[ Backslash \\ -%% Right bracket \] Circumflex \^ Underscore \_ -%% Grave accent \` Left brace \{ Vertical bar \| -%% Right brace \} Tilde \~} -\NeedsTeXFormat{LaTeX2e}[1995/12/01] -\ProvidesClass{manual} - [2012/04/23 v1.0 Akantu Manual LaTeX class (based on book.cls)] -\newcommand\@ptsize{} -\newif\if@restonecol -\newif\if@titlepage -\@titlepagetrue -\newif\if@openright -\newif\if@mainmatter \@mainmattertrue -\if@compatibility\else -\DeclareOption{a4paper} - {\setlength\paperheight {297mm}% - \setlength\paperwidth {210mm}} -\DeclareOption{a5paper} - {\setlength\paperheight {210mm}% - \setlength\paperwidth {148mm}} -\DeclareOption{b5paper} - {\setlength\paperheight {250mm}% - \setlength\paperwidth {176mm}} -\DeclareOption{letterpaper} - {\setlength\paperheight {11in}% - \setlength\paperwidth {8.5in}} -\DeclareOption{legalpaper} - {\setlength\paperheight {14in}% - \setlength\paperwidth {8.5in}} -\DeclareOption{executivepaper} - {\setlength\paperheight {10.5in}% - \setlength\paperwidth {7.25in}} -\DeclareOption{landscape} - {\setlength\@tempdima {\paperheight}% - \setlength\paperheight {\paperwidth}% - \setlength\paperwidth {\@tempdima}} -\fi -\if@compatibility - \renewcommand\@ptsize{0} -\else -\DeclareOption{10pt}{\renewcommand\@ptsize{0}} -\fi -\DeclareOption{11pt}{\renewcommand\@ptsize{1}} -\DeclareOption{12pt}{\renewcommand\@ptsize{2}} -\if@compatibility\else -\DeclareOption{oneside}{\@twosidefalse \@mparswitchfalse} -\fi -\DeclareOption{twoside}{\@twosidetrue \@mparswitchtrue} -\DeclareOption{draft}{\setlength\overfullrule{5pt}} -\if@compatibility\else -\DeclareOption{final}{\setlength\overfullrule{0pt}} -\fi -\DeclareOption{titlepage}{\@titlepagetrue} -\if@compatibility\else -\DeclareOption{notitlepage}{\@titlepagefalse} -\fi -\if@compatibility -\@openrighttrue -\else -\DeclareOption{openright}{\@openrighttrue} -\DeclareOption{openany}{\@openrightfalse} -\fi -\if@compatibility\else -\DeclareOption{onecolumn}{\@twocolumnfalse} -\fi -\DeclareOption{twocolumn}{\@twocolumntrue} -\DeclareOption{leqno}{\input{leqno.clo}} -\DeclareOption{fleqn}{\input{fleqn.clo}} -\DeclareOption{openbib}{% - \AtEndOfPackage{% - \renewcommand\@openbib@code{% - \advance\leftmargin\bibindent - \itemindent -\bibindent - \listparindent \itemindent - \parsep \z@ - }% - \renewcommand\newblock{\par}}% -} -\ExecuteOptions{a4paper,11pt,twoside,onecolumn,final,openright} -\ProcessOptions -\input{bk1\@ptsize.clo} -\setlength\lineskip{1\p@} -\setlength\normallineskip{1\p@} -\renewcommand\baselinestretch{} -\setlength\parskip{0\p@ \@plus \p@} -\@lowpenalty 51 -\@medpenalty 151 -\@highpenalty 301 -\setcounter{topnumber}{2} -\renewcommand\topfraction{.7} -\setcounter{bottomnumber}{1} -\renewcommand\bottomfraction{.3} -\setcounter{totalnumber}{3} -\renewcommand\textfraction{.2} -\renewcommand\floatpagefraction{.5} -\setcounter{dbltopnumber}{2} -\renewcommand\dbltopfraction{.7} -\renewcommand\dblfloatpagefraction{.5} -\if@twoside - \def\ps@headings{% - \let\@oddfoot\@empty\let\@evenfoot\@empty - \def\@evenhead{\thepage\hfil\slshape\leftmark}% - \def\@oddhead{{\slshape\rightmark}\hfil\thepage}% - \let\@mkboth\markboth - \def\chaptermark##1{% - \markboth {\MakeUppercase{% - \ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \@chapapp\ \thechapter. \ % - \fi - \fi - ##1}}{}}% - \def\sectionmark##1{% - \markright {\MakeUppercase{% - \ifnum \c@secnumdepth >\z@ - \thesection. \ % - \fi - ##1}}}} -\else - \def\ps@headings{% - \let\@oddfoot\@empty - \def\@oddhead{{\slshape\rightmark}\hfil\thepage}% - \let\@mkboth\markboth - \def\chaptermark##1{% - \markright {\MakeUppercase{% - \ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \@chapapp\ \thechapter. \ % - \fi - \fi - ##1}}}} -\fi -\def\ps@myheadings{% - \let\@oddfoot\@empty\let\@evenfoot\@empty - \def\@evenhead{\thepage\hfil\slshape\leftmark}% - \def\@oddhead{{\slshape\rightmark}\hfil\thepage}% - \let\@mkboth\@gobbletwo - \let\chaptermark\@gobble - \let\sectionmark\@gobble - } - \if@titlepage - \newcommand\maketitle{\begin{titlepage}% - \let\footnotesize\small - \let\footnoterule\relax - \let \footnote \thanks - \null\vfil - \vskip 60\p@ - \begin{center}% - {\LARGE \@title \par}% - \vskip 3em% - {\large - \lineskip .75em% - \begin{tabular}[t]{c}% - \@author - \end{tabular}\par}% - \vskip 1.5em% - {\large \@date \par}% % Set date in \large size. - \end{center}\par - \@thanks - \vfil\null - \end{titlepage}% - \setcounter{footnote}{0}% - \global\let\thanks\relax - \global\let\maketitle\relax - \global\let\@thanks\@empty - \global\let\@author\@empty - \global\let\@date\@empty - \global\let\@title\@empty - \global\let\title\relax - \global\let\author\relax - \global\let\date\relax - \global\let\and\relax -} -\else -\newcommand\maketitle{\par - \begingroup - \renewcommand\thefootnote{\@fnsymbol\c@footnote}% - \def\@makefnmark{\rlap{\@textsuperscript{\normalfont\@thefnmark}}}% - \long\def\@makefntext##1{\parindent 1em\noindent - \hb@xt@1.8em{% - \hss\@textsuperscript{\normalfont\@thefnmark}}##1}% - \if@twocolumn - \ifnum \col@number=\@ne - \@maketitle - \else - \twocolumn[\@maketitle]% - \fi - \else - \newpage - \global\@topnum\z@ % Prevents figures from going at top of page. - \@maketitle - \fi - \thispagestyle{plain}\@thanks - \endgroup - \setcounter{footnote}{0}% - \global\let\thanks\relax - \global\let\maketitle\relax - \global\let\@maketitle\relax - \global\let\@thanks\@empty - \global\let\@author\@empty - \global\let\@date\@empty - \global\let\@title\@empty - \global\let\title\relax - \global\let\author\relax - \global\let\date\relax - \global\let\and\relax -} -\def\@maketitle{% - \newpage - \null - \vskip 2em% - \begin{center}% - \let \footnote \thanks - {\LARGE \@title \par}% - \vskip 1.5em% - {\large - \lineskip .5em% - \begin{tabular}[t]{c}% - \@author - \end{tabular}\par}% - \vskip 1em% - {\large \@date}% - \end{center}% - \par - \vskip 1.5em} -\fi -\newcommand*\chaptermark[1]{} -\setcounter{secnumdepth}{2} -\newcounter {part} -\newcounter {chapter} -\newcounter {section}[chapter] -\newcounter {subsection}[section] -\newcounter {subsubsection}[subsection] -\newcounter {paragraph}[subsubsection] -\newcounter {subparagraph}[paragraph] -\renewcommand \thepart {\@Roman\c@part} -\renewcommand \thechapter {\@arabic\c@chapter} -\renewcommand \thesection {\thechapter.\@arabic\c@section} -\renewcommand\thesubsection {\thesection.\@arabic\c@subsection} -\renewcommand\thesubsubsection{\thesubsection.\@arabic\c@subsubsection} -\renewcommand\theparagraph {\thesubsubsection.\@arabic\c@paragraph} -\renewcommand\thesubparagraph {\theparagraph.\@arabic\c@subparagraph} -\newcommand\@chapapp{\chaptername} -\newcommand\frontmatter{% - \cleardoublepage - \@mainmatterfalse - \pagenumbering{roman}} -\newcommand\mainmatter{% - \cleardoublepage - \@mainmattertrue - \pagenumbering{arabic}} -\newcommand\backmatter{% - \if@openright - \cleardoublepage - \else - \clearpage - \fi - \@mainmatterfalse} -\newcommand\part{% - \if@openright - \cleardoublepage - \else - \clearpage - \fi - \thispagestyle{plain}% - \if@twocolumn - \onecolumn - \@tempswatrue - \else - \@tempswafalse - \fi - \null\vfil - \secdef\@part\@spart} - -\def\@part[#1]#2{% - \ifnum \c@secnumdepth >-2\relax - \refstepcounter{part}% - \addcontentsline{toc}{part}{\thepart\hspace{1em}#1}% - \else - \addcontentsline{toc}{part}{#1}% - \fi - \markboth{}{}% - {\centering - \interlinepenalty \@M - \normalfont - \ifnum \c@secnumdepth >-2\relax - \huge\bfseries \partname\nobreakspace\thepart - \par - \vskip 20\p@ - \fi - \Huge \bfseries #2\par}% - \@endpart} -\def\@spart#1{% - {\centering - \interlinepenalty \@M - \normalfont - \Huge \bfseries #1\par}% - \@endpart} -\def\@endpart{\vfil\newpage - \if@twoside - \if@openright - \null - \thispagestyle{empty}% - \newpage - \fi - \fi - \if@tempswa - \twocolumn - \fi} -\newcommand\chapter{\if@openright\cleardoublepage\else\clearpage\fi - \thispagestyle{plain}% - \global\@topnum\z@ - \@afterindentfalse - \secdef\@chapter\@schapter} -\def\@chapter[#1]#2{\ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \refstepcounter{chapter}% - \typeout{\@chapapp\space\thechapter.}% - \addcontentsline{toc}{chapter}% - {\protect\numberline{\thechapter}#1}% - \else - \addcontentsline{toc}{chapter}{#1}% - \fi - \else - \addcontentsline{toc}{chapter}{#1}% - \fi - \chaptermark{#1}% - \addtocontents{lof}{\protect\addvspace{10\p@}}% - \addtocontents{lot}{\protect\addvspace{10\p@}}% - \if@twocolumn - \@topnewpage[\@makechapterhead{#2}]% - \else - \@makechapterhead{#2}% - \@afterheading - \fi} -\def\@makechapterhead#1{% - \vspace*{50\p@}% - {\parindent \z@ \raggedright \normalfont - \ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \huge\bfseries \@chapapp\space \thechapter - \par\nobreak - \vskip 20\p@ - \fi - \fi - \interlinepenalty\@M - \Huge \bfseries #1\par\nobreak - \vskip 40\p@ - }} -\def\@schapter#1{\if@twocolumn - \@topnewpage[\@makeschapterhead{#1}]% - \else - \@makeschapterhead{#1}% - \@afterheading - \fi} -\def\@makeschapterhead#1{% - \vspace*{50\p@}% - {\parindent \z@ \raggedright - \normalfont - \interlinepenalty\@M - \Huge \bfseries #1\par\nobreak - \vskip 40\p@ - }} -\newcommand\section{\@startsection {section}{1}{\z@}% - {-3.5ex \@plus -1ex \@minus -.2ex}% - {2.3ex \@plus.2ex}% - {\normalfont\Large\bfseries}} -\newcommand\subsection{\@startsection{subsection}{2}{\z@}% - {-3.25ex\@plus -1ex \@minus -.2ex}% - {1.5ex \@plus .2ex}% - {\normalfont\large\bfseries}} -\newcommand\subsubsection{\@startsection{subsubsection}{3}{\z@}% - {-3.25ex\@plus -1ex \@minus -.2ex}% - {1.5ex \@plus .2ex}% - {\normalfont\normalsize\bfseries}} -\newcommand\paragraph{\@startsection{paragraph}{4}{\z@}% - {3.25ex \@plus1ex \@minus.2ex}% - {-1em}% - {\normalfont\normalsize\bfseries}} -\newcommand\subparagraph{\@startsection{subparagraph}{5}{\parindent}% - {3.25ex \@plus1ex \@minus .2ex}% - {-1em}% - {\normalfont\normalsize\bfseries}} -\if@twocolumn - \setlength\leftmargini {2em} -\else - \setlength\leftmargini {2.5em} -\fi -\leftmargin \leftmargini -\setlength\leftmarginii {2.2em} -\setlength\leftmarginiii {1.87em} -\setlength\leftmarginiv {1.7em} -\if@twocolumn - \setlength\leftmarginv {.5em} - \setlength\leftmarginvi {.5em} -\else - \setlength\leftmarginv {1em} - \setlength\leftmarginvi {1em} -\fi -\setlength \labelsep {.5em} -\setlength \labelwidth{\leftmargini} -\addtolength\labelwidth{-\labelsep} -\@beginparpenalty -\@lowpenalty -\@endparpenalty -\@lowpenalty -\@itempenalty -\@lowpenalty -\renewcommand\theenumi{\@arabic\c@enumi} -\renewcommand\theenumii{\@alph\c@enumii} -\renewcommand\theenumiii{\@roman\c@enumiii} -\renewcommand\theenumiv{\@Alph\c@enumiv} -\newcommand\labelenumi{\theenumi.} -\newcommand\labelenumii{(\theenumii)} -\newcommand\labelenumiii{\theenumiii.} -\newcommand\labelenumiv{\theenumiv.} -\renewcommand\p@enumii{\theenumi} -\renewcommand\p@enumiii{\theenumi(\theenumii)} -\renewcommand\p@enumiv{\p@enumiii\theenumiii} -\newcommand\labelitemi{\textbullet} -\newcommand\labelitemii{\normalfont\bfseries \textendash} -\newcommand\labelitemiii{\textasteriskcentered} -\newcommand\labelitemiv{\textperiodcentered} -\newenvironment{description} - {\list{}{\labelwidth\z@ \itemindent-\leftmargin - \let\makelabel\descriptionlabel}} - {\endlist} -\newcommand*\descriptionlabel[1]{\hspace\labelsep - \normalfont\bfseries #1} -\newenvironment{verse} - {\let\\\@centercr - \list{}{\itemsep \z@ - \itemindent -1.5em% - \listparindent\itemindent - \rightmargin \leftmargin - \advance\leftmargin 1.5em}% - \item\relax} - {\endlist} -\newenvironment{quotation} - {\list{}{\listparindent 1.5em% - \itemindent \listparindent - \rightmargin \leftmargin - \parsep \z@ \@plus\p@}% - \item\relax} - {\endlist} -\newenvironment{quote} - {\list{}{\rightmargin\leftmargin}% - \item\relax} - {\endlist} -\if@compatibility -\newenvironment{titlepage} - {% - \cleardoublepage - \if@twocolumn - \@restonecoltrue\onecolumn - \else - \@restonecolfalse\newpage - \fi - \thispagestyle{empty}% - \setcounter{page}\z@ - }% - {\if@restonecol\twocolumn \else \newpage \fi - } -\else -\newenvironment{titlepage} - {% - \cleardoublepage - \if@twocolumn - \@restonecoltrue\onecolumn - \else - \@restonecolfalse\newpage - \fi - \thispagestyle{empty}% - \setcounter{page}\@ne - }% - {\if@restonecol\twocolumn \else \newpage \fi - \if@twoside\else - \setcounter{page}\@ne - \fi - } -\fi -\newcommand\appendix{\par - \setcounter{chapter}{0}% - \setcounter{section}{0}% - \gdef\@chapapp{\appendixname}% - \gdef\thechapter{\@Alph\c@chapter}} -\setlength\arraycolsep{5\p@} -\setlength\tabcolsep{6\p@} -\setlength\arrayrulewidth{.4\p@} -\setlength\doublerulesep{2\p@} -\setlength\tabbingsep{\labelsep} -\skip\@mpfootins = \skip\footins -\setlength\fboxsep{3\p@} -\setlength\fboxrule{.4\p@} -\@addtoreset {equation}{chapter} -\renewcommand\theequation - {\ifnum \c@chapter>\z@ \thechapter.\fi \@arabic\c@equation} -\newcounter{figure}[chapter] -\renewcommand \thefigure - {\ifnum \c@chapter>\z@ \thechapter.\fi \@arabic\c@figure} -\def\fps@figure{tbp} -\def\ftype@figure{1} -\def\ext@figure{lof} -\def\fnum@figure{\figurename\nobreakspace\thefigure} -\newenvironment{figure} - {\@float{figure}} - {\end@float} -\newenvironment{figure*} - {\@dblfloat{figure}} - {\end@dblfloat} -\newcounter{table}[chapter] -\renewcommand \thetable - {\ifnum \c@chapter>\z@ \thechapter.\fi \@arabic\c@table} -\def\fps@table{tbp} -\def\ftype@table{2} -\def\ext@table{lot} -\def\fnum@table{\tablename\nobreakspace\thetable} -\newenvironment{table} - {\@float{table}} - {\end@float} -\newenvironment{table*} - {\@dblfloat{table}} - {\end@dblfloat} -\newlength\abovecaptionskip -\newlength\belowcaptionskip -\setlength\abovecaptionskip{10\p@} -\setlength\belowcaptionskip{0\p@} -\long\def\@makecaption#1#2{% - \vskip\abovecaptionskip - \sbox\@tempboxa{#1: #2}% - \ifdim \wd\@tempboxa >\hsize - #1: #2\par - \else - \global \@minipagefalse - \hb@xt@\hsize{\hfil\box\@tempboxa\hfil}% - \fi - \vskip\belowcaptionskip} -\DeclareOldFontCommand{\rm}{\normalfont\rmfamily}{\mathrm} -\DeclareOldFontCommand{\sf}{\normalfont\sffamily}{\mathsf} -\DeclareOldFontCommand{\tt}{\normalfont\ttfamily}{\mathtt} -\DeclareOldFontCommand{\bf}{\normalfont\bfseries}{\mathbf} -\DeclareOldFontCommand{\it}{\normalfont\itshape}{\mathit} -\DeclareOldFontCommand{\sl}{\normalfont\slshape}{\@nomath\sl} -\DeclareOldFontCommand{\sc}{\normalfont\scshape}{\@nomath\sc} -\DeclareRobustCommand*\cal{\@fontswitch\relax\mathcal} -\DeclareRobustCommand*\mit{\@fontswitch\relax\mathnormal} -\newcommand\@pnumwidth{1.55em} -\newcommand\@tocrmarg{2.55em} -\newcommand\@dotsep{4.5} -\setcounter{tocdepth}{2} -\newcommand\tableofcontents{% - \if@twocolumn - \@restonecoltrue\onecolumn - \else - \@restonecolfalse - \fi - \chapter*{\contentsname - \@mkboth{% - \MakeUppercase\contentsname}{\MakeUppercase\contentsname}}% - \pdfbookmark[0]{\contentsname}{} - \@starttoc{toc}% - \if@restonecol\twocolumn\fi - } -\newcommand*\l@part[2]{% - \ifnum \c@tocdepth >-2\relax - \addpenalty{-\@highpenalty}% - \addvspace{2.25em \@plus\p@}% - \setlength\@tempdima{3em}% - \begingroup - \parindent \z@ \rightskip \@pnumwidth - \parfillskip -\@pnumwidth - {\leavevmode - \large \bfseries #1\hfil \hb@xt@\@pnumwidth{\hss #2}}\par - \nobreak - \global\@nobreaktrue - \everypar{\global\@nobreakfalse\everypar{}}% - \endgroup - \fi} -\newcommand*\l@chapter[2]{% - \ifnum \c@tocdepth >\m@ne - \addpenalty{-\@highpenalty}% - \vskip 1.0em \@plus\p@ - \setlength\@tempdima{1.5em}% - \begingroup - \parindent \z@ \rightskip \@pnumwidth - \parfillskip -\@pnumwidth - \leavevmode \bfseries - \advance\leftskip\@tempdima - \hskip -\leftskip - #1\nobreak\hfil \nobreak\hb@xt@\@pnumwidth{\hss #2}\par - \penalty\@highpenalty - \endgroup - \fi} -\newcommand*\l@section{\@dottedtocline{1}{1.5em}{2.3em}} -\newcommand*\l@subsection{\@dottedtocline{2}{3.8em}{3.2em}} -\newcommand*\l@subsubsection{\@dottedtocline{3}{7.0em}{4.1em}} -\newcommand*\l@paragraph{\@dottedtocline{4}{10em}{5em}} -\newcommand*\l@subparagraph{\@dottedtocline{5}{12em}{6em}} -\newcommand\listoffigures{% - \if@twocolumn - \@restonecoltrue\onecolumn - \else - \@restonecolfalse - \fi - \chapter*{\listfigurename}% - \@mkboth{\MakeUppercase\listfigurename}% - {\MakeUppercase\listfigurename}% - \@starttoc{lof}% - \if@restonecol\twocolumn\fi - } -\newcommand*\l@figure{\@dottedtocline{1}{1.5em}{2.3em}} -\newcommand\listoftables{% - \if@twocolumn - \@restonecoltrue\onecolumn - \else - \@restonecolfalse - \fi - \chapter*{\listtablename}% - \@mkboth{% - \MakeUppercase\listtablename}% - {\MakeUppercase\listtablename}% - \@starttoc{lot}% - \if@restonecol\twocolumn\fi - } -\let\l@table\l@figure -\newdimen\bibindent -\setlength\bibindent{1.5em} -\newenvironment{thebibliography}[1] - {\chapter*{\bibname}% - \@mkboth{\MakeUppercase\bibname}{\MakeUppercase\bibname}% - \list{\@biblabel{\@arabic\c@enumiv}}% - {\settowidth\labelwidth{\@biblabel{#1}}% - \leftmargin\labelwidth - \advance\leftmargin\labelsep - \@openbib@code - \usecounter{enumiv}% - \let\p@enumiv\@empty - \renewcommand\theenumiv{\@arabic\c@enumiv}}% - \sloppy - \clubpenalty4000 - \@clubpenalty \clubpenalty - \widowpenalty4000% - \sfcode`\.\@m} - {\def\@noitemerr - {\@latex@warning{Empty `thebibliography' environment}}% - \endlist} -\newcommand\newblock{\hskip .11em\@plus.33em\@minus.07em} -\let\@openbib@code\@empty -\newenvironment{theindex} - {\if@twocolumn - \@restonecolfalse - \else - \@restonecoltrue - \fi - \twocolumn[\@makeschapterhead{\indexname}]% - \@mkboth{\MakeUppercase\indexname}% - {\MakeUppercase\indexname}% - \thispagestyle{plain}\parindent\z@ - \parskip\z@ \@plus .3\p@\relax - \columnseprule \z@ - \columnsep 35\p@ - \let\item\@idxitem} - {\if@restonecol\onecolumn\else\clearpage\fi} -\newcommand\@idxitem{\par\hangindent 40\p@} -\newcommand\subitem{\@idxitem \hspace*{20\p@}} -\newcommand\subsubitem{\@idxitem \hspace*{30\p@}} -\newcommand\indexspace{\par \vskip 10\p@ \@plus5\p@ \@minus3\p@\relax} -\renewcommand\footnoterule{% - \kern-3\p@ - \hrule\@width.4\columnwidth - \kern2.6\p@} -\@addtoreset{footnote}{chapter} -\newcommand\@makefntext[1]{% - \parindent 1em% - \noindent - \hb@xt@1.8em{\hss\@makefnmark}#1} -\newcommand\contentsname{Contents} -\newcommand\listfigurename{List of Figures} -\newcommand\listtablename{List of Tables} -\newcommand\bibname{Bibliography} -\newcommand\indexname{Index} -\newcommand\figurename{Figure} -\newcommand\tablename{Table} -\newcommand\partname{Part} -\newcommand\chaptername{Chapter} -\newcommand\appendixname{Appendix} -\def\today{\ifcase\month\or - January\or February\or March\or April\or May\or June\or - July\or August\or September\or October\or November\or December\fi - \space\number\day, \number\year} -\setlength\columnsep{10\p@} -\setlength\columnseprule{0\p@} -\pagestyle{headings} -\pagenumbering{arabic} -\if@twoside -\else - \raggedbottom -\fi -\if@twocolumn - \twocolumn - \sloppy - \flushbottom -\else - \onecolumn -\fi -\endinput -%% -%% End of file `manual.cls'. diff --git a/doc/manual.sty b/doc/manual.sty deleted file mode 100644 index af1ec62..0000000 --- a/doc/manual.sty +++ /dev/null @@ -1,305 +0,0 @@ -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% %% -%% LaTeX STYLE SHEET for BlackDynamite DOCUMENTATION %% -%% %% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -% Geometry -\usepackage{a4wide} -\usepackage{geometry} -\geometry{ - pdftex=true, - twoside=true, - margin=20mm, - bottom=20mm, - top=20mm, - bindingoffset=5.5mm -} - -% Font encoding -\usepackage[T1]{fontenc} -\usepackage{palatino} -\usepackage{xspace} -% Line spacing -\linespread{1.05}\selectfont - -% Allow spaces to be added at the end of macro -\usepackage{xspace} - -% Mathematics (including correct font encoding, after amsmath) -\usepackage{amsmath} -\usepackage{amssymb} -\usepackage{pxfonts} -%\setlength\mathindent{2em} - -% Some unit stuff -\usepackage[squaren]{SIunits} - -% Easy tables -\usepackage{booktabs} -\usepackage{longtable} -\usepackage{rotating} % For sideways tables -\usepackage{multirow} % For larger cells - -% A special environment for element tables -\newcommand\elemline{\hline}%{\noalign{\smallskip}\hline\noalign{\smallskip}} -\newcommand\elemcooroned{\ensuremath{\left(\xi\right)}} -\newcommand\elemcoortwod{\ensuremath{\left(\xi\;,\;\eta\right)}} -\newcommand\elemcoorthreed{\ensuremath{\left(\xi\;,\;\eta\;,\;\zeta\right)}} -\newcommand\elemdshapeoned{\ensuremath{\left(\partial N_i/\partial\xi\right)}} -\newcommand\elemdshapetwod{\ensuremath{\left(\partial N_i/\partial\xi\;,\;\partial N_i/\partial\eta\right)}} -\newcommand\elemdshapethreed{\ensuremath{\left(\partial N_i/\partial\xi\;,\;\partial N_i/\partial\eta\;,\;\partial N_i/\partial\zeta\right)}} -\newcommand\inelemone[1]{\ensuremath{#1}} -%\newcommand\inelemtwo[2]{\ensuremath{\begin{pmatrix} {\; #1} & \!,\! & {#2 \;} \end{pmatrix}}} -%\newcommand\inelemthree[3]{\ensuremath{\begin{pmatrix} {\; #1} & \!,\! & {#2} & \!,\! & {#3 \;} \end{pmatrix}}} -\newcommand\inelemtwo[2]{\ensuremath{\left( {\; #1} \; , \; {#2 \;} \right)}} -\newcommand\inelemthree[3]{\ensuremath{\left( {\; #1} \; , \; {#2} \; , \; {#3 \;} \right)}} -\newcommand\inquadone[1]{\ensuremath{#1}} -\newcommand\inquadtwo[2]{\ensuremath{\left(\, #1 \, , \, #2 \,\right)}} -\newcommand\inquadthree[3]{\ensuremath{\left(\, #1 \, , \, #2 \, , \, #3 \,\right)}} -%\newcommand\quada{\tfrac{1}{20}\left(5-\sqrt{5}\right)} -%\newcommand\quadb{\tfrac{1}{20}\left(5+3\sqrt{5}\right)} -\newcommand\quada{\tfrac{\left(5-\sqrt{5}\right)}{20}} -\newcommand\quadb{\tfrac{\left(5+3\sqrt{5}\right)}{20}} -\newenvironment{Element}[1] - {\begin{table*}[!htbp]\footnotesize - \ifthenelse{\equal{#1}{1D}}{\renewcommand{\arraystretch}{1.50}}{} - \ifthenelse{\equal{#1}{2D}}{\renewcommand{\arraystretch}{1.60}}{} - \ifthenelse{\equal{#1}{3D}}{\renewcommand{\arraystretch}{1.70}}{} - \textbf{{\normalsize Element properties}}\newline\vspace*{0.5\baselineskip} - \begin{tabular}{c|c|c|c} - %\multicolumn{4}{l}{\textbf{{\normalsize Element properties}}} \\ - Node ($i$) & Coord. - \ifthenelse{\equal{#1}{1D}}{\elemcooroned}{} - \ifthenelse{\equal{#1}{2D}}{\elemcoortwod}{} - \ifthenelse{\equal{#1}{3D}}{\elemcoorthreed}{} - & Shape function ($N_{i}$) - & {Derivative - \ifthenelse{\equal{#1}{1D}}{\elemdshapeoned}{} - \ifthenelse{\equal{#1}{2D}}{\elemdshapetwod}{} - \ifthenelse{\equal{#1}{3D}}{\elemdshapethreed}{} - }\\ - \elemline - } - {\end{tabular}\end{table*}} -\newenvironment{QuadPoints}%[1] - {\vspace*{-\baselineskip} - \begin{table*}[!htbp]\footnotesize - \renewcommand{\arraystretch}{1.50} - \textbf{{\normalsize Gaussian quadrature points}}\newline\vspace*{0.5\baselineskip} - } - {\end{table*}} - -% Nice coloring -%\usepackage[dvipsnames,usenames,table]{xcolor} -\usepackage[dvipsnames]{xcolor} -\definecolor{RED}{rgb}{1,0,0} -\definecolor{cppbg}{HTML}{EBF2F2} -\definecolor{shellbg}{HTML}{F5EDE4} -\definecolor{commentcolor}{HTML}{101280} - -% Allow for the use of listings -\usepackage{listings} - -% Create an index -\usepackage{makeidx} - -% Figure handling -\usepackage{graphics} -\usepackage{epsfig} -\usepackage[lofdepth,lotdepth]{subfig} -\usepackage{tikz} -\usetikzlibrary{decorations} -\renewcommand{\floatpagefraction}{.6} % default: .5 -\renewcommand\topfraction{0.9} % 90% of page top can be a float (Standard 0.7) -\renewcommand\bottomfraction{0.1} % 10% of page bottom can be a float (Standard 0.3) -\renewcommand\textfraction{0.1} % only 10% of page must to be text (Standard 0.2) - -% Removes parenthese around subfig number -\renewcommand*{\thesubfigure}{\alph{subfigure}} - -% Create a new list style for C++ -\lstdefinestyle{C++}{ - language=C++, % the language of the code - basicstyle=\small\ttfamily, % Without beramono, we'd get cmtt, the teletype font. - commentstyle=\color{commentcolor}\itshape, - keywordstyle=\color{DarkOrchid}\bfseries, - % fontadjust, - % numbers=left, % where to put the line-numbers - % numberstyle=\tiny, % the size of the fonts that are used for the line-numbers - % stepnumber=2, % the step between two line-numbers. If it's 1, each line will - % be numbered - % numbersep=5pt, % how far the line-numbers are from the code - % showspaces=false, % show spaces adding particular underscores - showstringspaces=false, % underline spaces within strings - % showtabs=false, % show tabs within strings adding particular underscores - % frame=llines, % adds a frame around the code - % frame=tb, - tabsize=2, % sets default tabsize to 2 spaces - captionpos=b, % sets the caption-position to bottom - breaklines=true, % sets automatic line breaking - breakatwhitespace=false, % sets if automatic breaks should only happen at - % whitespace - % title=\lstname, % show the filename of files included with \lstinputlisting; - % also try caption instead of title - % escapeinside={\%*}{*)}, % if you want to add a comment within your code - xleftmargin=1cm, - xrightmargin=1cm, - mathescape=true, - escapechar=\%, - morekeywords={Real, UInt, Int}, - columns=flexible, - keepspaces=true, - backgroundcolor=\color{cppbg} -} - -% Create new list style for the shell -\lstdefinestyle{shell}{ - language=bash, % the language of the code - basicstyle=\scriptsize\ttfamily, % Without beramono, we'd get cmtt, the teletype font. - showstringspaces=false, % underline spaces within strings - tabsize=2, % sets default tabsize to 2 spaces - captionpos=b, % sets the caption-position to bottom - breaklines=true, % sets automatic line breaking - breakatwhitespace=false, - xleftmargin=1cm, - xrightmargin=1cm, - escapechar=\%, - morekeywords={mkdir, make, ccmake, cmake}, - columns=flexible, - keepspaces=true, - backgroundcolor=\color{shellbg} -} - -\lstdefinelanguage{lmconfig} - {morekeywords={Section,endSection,LET,GEOMETRY,FILTER,KEYWORD,PRINT,FOR,endFOR,range, in}, - sensitive=true, - morecomment=[l]{\#}, - } - -\lstdefinestyle{config}{ - language=lmconfig, % the language of the code - basicstyle=\small\ttfamily, % Without beramono, we'd get cmtt, the teletype font. - commentstyle=\color{commentcolor}\itshape, - keywordstyle=\color{DarkOrchid}\bfseries, - showstringspaces=false, % underline spaces within strings - tabsize=2, % sets default tabsize to 2 spaces - captionpos=b, % sets the caption-position to bottom - breaklines=true, % sets automatic line breaking - breakatwhitespace=false, - xleftmargin=1cm, - xrightmargin=1cm, - escapechar=\%, - morekeywords={mkdir, make, ccmake, cmake}, - columns=flexible, - keepspaces=true, - backgroundcolor=\color{cppbg} -} - -% Set some derived listing environments -\lstnewenvironment{cpp}{\lstset{style=C++}}{} -\lstnewenvironment{lmconfig}{\lstset{style=config}}{} -\lstnewenvironment{command}{\lstset{style=shell}}{} - -% Make sure outputspace is white -\makeatletter -\def\lst@outputspace{{\ifx\lst@bkgcolor\empty\color{white}\else\lst@bkgcolor\fi\lst@visiblespace}} -\makeatother - -% Renow a label in the itemized lists -\renewcommand{\labelitemi}{$\mathbf{\circ}$} - -% Don't care so much about overfull h-boxes -\sloppy - -% Penalty adjusments -%\widowpenalty=10000 % Single lines/word on beginning of page -%\clubpenalty=10000 % Single lines/word at end of page -%\hyphenpenalty=2000 % Hyphenate words -%\tolerance=250 % To adjust the hyphenation of words, increase the tolerance to discourage hyphenation - % the higher the value, the more ugly the gaps between words - % default \tolerance=200 -%\hfuzz=10000pt % threshold when an overfull hbox is reported, default \hfuzz=0.1pt -%\vfuzz=10000pt % threshold to report an overfull vbox, default \vfuzz=0.1pt -%\hbadness=10000 % threshold to report an underfull \hbox -%\vbadness=10000 % threshold to report an underfull \vbox protokolliert wird. -\emergencystretch=0pt % causes a third attempt to fix bad paragraphs and defines a maximum limit to stretch them - -% Insert an empty or a blank page -\newcommand{\insertemptypage}{\newpage\hbox{}\newpage} -\newcommand{\insertblankpage}{\newpage\thispagestyle{empty}\hbox{}\newpage} - -% No page number on an empty page -\let\origdoublepage\cleardoublepage -\newcommand{\clearemptydoublepage}{% - \clearpage - {\thispagestyle{empty}\origdoublepage}% -} -\let\cleardoublepage\clearemptydoublepage - -% A new ruler for in chapters -\newcommand\InChapterRule{\addvspace{\baselineskip}\rule{0.3\linewidth}{0.25pt}} - -% New footnote style -%\def\@fnsymbol#1{\ifcase#1\or *\or \dagger\or \ddagger\or \mathchar "278\or \mathchar "27B\or \|\or **\or \dagger\dagger \or \ddagger\ddagger \else\@ctrerr\fi\relax} -\def\@fnsymbol#1{*\xspace\relax} -\renewcommand{\thefootnote}{\fnsymbol{footnote}} % Symbols rather than numbers -\def\footnoterule{\vspace*{0.5\baselineskip}\InChapterRule\vspace*{0.25\baselineskip}} - -% Improved look of the Table of Contents -\usepackage[nottoc,notbib]{tocbibind} -\usepackage[dotinlabels]{titletoc} -\titlecontents{chapter}[1.4pc] - {\addvspace{0.6pc}\large\bfseries\filright} - {\contentslabel[\thecontentslabel.]{1.4pc}} - {\hspace{-1.4pc}} - {\hfill\contentspage} - [\addvspace{2pt}] -\titlecontents{section}[3.4pc] - {\filright} - {\contentslabel[\thecontentslabel]{2pc}} - {\hspace{-2pc}} - {\titlerule*[6pt]{.}\contentspage} - [] -\titlecontents{subsection}[5.0pc] - {\filright} - {\contentslabel[\thecontentslabel]{2.4pc}} - {} - {\titlerule*[6pt]{.}\contentspage} - [] -\setcounter{tocdepth}{2} -\newcommand\addspaceintoc{\addtocontents{toc}{\protect\addvspace{20pt}}} - -% Change the appearance of the bibliography -%\bibliographystyle{manual-bibliographystyle} -\renewcommand\bibname{References} -\usepackage{cite} % To sort citations: [2,10-14] -\let\oldthebibliography=\thebibliography -\let\endoldthebibliography=\endthebibliography -\renewenvironment{thebibliography}[1] -{ \begin{oldthebibliography}{#1} -% \small - \addcontentsline{toc}{chapter}{\bibname} - \setlength{\labelsep}{2mm} - \setlength{\parskip}{0\baselineskip} - \setlength{\itemsep}{0.24\baselineskip} -} -{ \end{oldthebibliography} } - - -% Hyperref -\usepackage{url} -\usepackage[pdftex, - bookmarks=true, - bookmarksnumbered=true, - linkbordercolor={1 1 1}, - pdfborder={0 0 0}, - pdfpagemode=UseOutlines -]{hyperref} -\hypersetup{ - pdfauthor={Computational Solid Mechanics Laboratory - EPFL}, - pdftitle={BlackDynamite User's Guide}, - pdfsubject={Parametric study manager} -} - - diff --git a/doc/manual.tex b/doc/manual.tex deleted file mode 100644 index 9f3122a..0000000 --- a/doc/manual.tex +++ /dev/null @@ -1,536 +0,0 @@ -\documentclass[openright,a4paper,9pt,fleqn]{manual} -\usepackage{manual} -\usepackage{manual-macros} - -\setlength{\oddsidemargin}{-1cm} % Marge gauche sur pages impaires -\setlength{\evensidemargin}{-1cm} % Marge gauche sur pages paires -\setlength{\marginparwidth}{0cm} % Largeur de note dans la marge -\setlength{\textwidth}{18cm} % Largeur de la zone de texte (17cm) -\setlength{\marginparsep}{0pt} % Separation de la marge - \setlength\parindent{0pt} - -\author{} -\date{} -\newcommand{\version}{0.1} - -\newcommand{\blackdynamite}{\textbf{BlackDynamite}\xspace} - -\title{\textbf{\Huge \blackdynamite}\\ - \vspace{0.5cm} - \textbf{\huge User's Guide}\\ - \vspace{1cm} - {\small \today{} --- Version \version} -} - - - - -\begin{document} - -\setcounter{page}{1} -\renewcommand{\thepage}{\roman{page}} - -\pdfbookmark[0]{Titlepage}{maintitlepage} -\label{maintitlepage} -\maketitle -\tableofcontents -\ifodd\value{page} \insertblankpage -\else \insertblankpage\insertblankpage \fi - -\setcounter{page}{1} -\renewcommand\thepage{\arabic{page}} - -\chapter{Installing and compiling} -\section{Prerequisites} -Under Debian/Ubuntu, you should install several packages in order to -use plainly BlackDynamite: - -\begin{command} -sudo apt-get install python-argcomplete python-psycopg2 -\end{command} - -\section{Setting up the PostGreSQL database} - -If you want to setup a PostGreSQL server to store BlackDynamite data, -then you have to follow this procedure. - -Install the PSQL server: -\begin{command} -sudo apt-get install postgresql-9.4 -\end{command} - -You know need privileges to create databases and users. -This can be done using the following: - -\begin{command} -bash:> sudo su postgres -bash:> psql - -postgres=# -\end{command} - -Then you should create a user: -\begin{command} -postgres=# create user mylogin; -\end{command} - -configure plpgsql language to the database: -\begin{command} -postgres=# CREATE PROCEDURAL LANGUAGE plpgsql; -\end{command} - -And a database associated with that user: -\begin{command} -postgres=# create database mylogin; -postgres=# alter database mylogin owner to mylogin; -postgres=# grant create on database mylogin TO mylogin; -postgres=# alter role mylogin with password 'my_pass'; -\end{command} - - - - - -\section{Installing completion} -To benefit the autocompletion for BlackDynamite the following steps are needed. -You first need to install the argcomplete modules. Either by typing (Depending of your Ubuntu/Debian version) : -\begin{command} -sudo apt-get install python-argcomplete -\end{command} -or: -\begin{command} -sudo apt-get install python-pip -sudo pip install argcomplete -\end{command} - -Then %activate-global-python-argcomplete -you must insert the following in your .bashrc - -\begin{command} -eval "$(register-python-argcomplete getRunInfo.py)" -eval "$(register-python-argcomplete launchRuns.py)" -eval "$(register-python-argcomplete canYouDigIt.py)" -eval "$(register-python-argcomplete cleanRuns.py)" -eval "$(register-python-argcomplete updateRuns.py)" -eval "$(register-python-argcomplete enterRun.py)" -eval "$(register-python-argcomplete enterRun.py)" -eval "$(register-python-argcomplete saveBDStudy.py)" -export PATH=$PATH:~/.../blackdynamite/bin/ -export PYTHONPATH=$PYTHONPATH:~/.../blackdynamite/python -\end{command} - - -\section{Register hosts to BlackDynamite} - -In the .blackdynamite folder (in your home) you should add the servers where your databases are, with the option and information of your choice. For each database you can add a file .bd of the name of the server (or an alias and specify the host inside: host = yourHost.domain.countryID). It is also recommended to specify the password of the database to avoid typing it when using autocompletion. - - -\chapter{Introduction and philosophy} - -Blackdynamite is merely a tool to help -achieving a few things: - -\begin{enumerate} -\item Launching a program repeatedly with varying parameters, to explore the - chosen parametric space. -\item Collect and sort results of \textbf{\color{red}Small sizes} benefiting - from the power of modern databases. -\item Analyse the results by making requests to the associated databases. -\end{enumerate} - -\paragraph{Launching} is made simple by allowing any executable -to be launched. The set of directories will be generated and managed -by BlackDynamite to prevent errors. Requests of any kind will then -be made to the underlying database through friendly commands of BlackDynamite. - -\paragraph{Collecting} the results will be possible thanks to the Blackdynamite C/C++ and python -API which will let you send results directly to the database and thus automatically sort them. -This is extremely useful. However heavy data such as Paraview files or any other kind of data -should not be pushed to the database fr obvious performance issues. - -\paragraph{Analysis} of the results can be made easy thanks to Blackdynamite which -can retrieve data information in the form of Numpy array to be used, analyzed or plotted -thanks to the powerful and vast Python libraries such as Matplotlib and Scipy. - - - -\chapter{Setting up a parametric study} - -\section{Chose the parameters of the study} -The first thing to do is to setup the table in the database associated -with the study we want to perform. For this to be done -you need, first of all, to list all the parameters that decide -a specific case/computation. This parameters can be of simple types -like string, integers, floats, etc. At current time no vectorial -quantity can be considered as an input parameter. -Once this list is done you need to create a script, usually named \underline{\code{createDB.py}} -that will do this task. Let us examine such an example script. - -\subsection{Setting up blackdynamite python modules} - -First we need to set the python headers and to import the \blackdynamite modules by -\begin{command} -#!/usr/bin/env python - -import BlackDynamite as BD -\end{command} - -Then you have to create a generic black dynamite parser -and parse the system (including the connection parameters and credentials) - -\begin{command} -parser = BD.BDParser() -params = parser.parseBDParameters() -\end{command} - -This mechanism allows to easily inherit from the parser mechanism -of BlackDynamite, including the completion (if activated: see installation instructions). -Then you can connect to the black dynamite database - -\begin{command} -base = BD.base.Base(**params) -\end{command} - - -\subsection{Setting up of the parametric space: the jobs pattern} - -Then you have to define the parametric space (at present time, -the parametric space cannot be changed once the study started: -be careful with your choices). -Any particular job is defined as a point in the parametric space. -For instance, to create a job description and add the parameters -with int, float or list parameters, you can use the following python sequence. - -\begin{command} -myjob_desc = BD.job.Job(base) - -myjob_desc.types["param1"] = int -myjob_desc.types["param2"] = float -myjob_desc.types["param3"] = str -\end{command} - -%\emph{Do not name your parameters with PostGreSQL keywords.} - -\subsection{Setting up of the run space} - -Aside of the jobs, a run will represent a particular realisation (computation) -of a job. To get clearer, the run will contain information of the machine it was run on, -the executable version, or the number of processors employed. -For instance creating the run pattern can be done with: -\begin{command} -myruns_desc = run.Run(base) - -myruns_desc.types["compiler"] = str -\end{command} - -There are default entries to the description of runs. -These are: -\begin{itemize} -\item machine\_name: the name of the machine where the run must be executed -\item job\_id (integer): the ID of the running job -\item has\_started (bool): flag to know whether the job has already started -\item has\_finished (bool): flag to know whether the job has already finished -\item run\_name (string): the name of the run -\item wait\_id (int): The id of a run to wait before starting -\item start\_time (TIMESTAMP): The start time for the run -\end{itemize} - -\subsection{Commit the changes to the database} - -Then you have to request for the creation of the database -\begin{command} -base.createBase(myjob_desc,myruns_desc,**params) -\end{command} - -You have to launch the script. As mentioned, all BlackDynamite -scripts inherit from the parsing system. So that when needing to launch -one of these codes, you can always claim for the valid keywords: -\begin{command} -./createDB.py --help - -usage: createDB.py [-h] [--job_constraints JOB_CONSTRAINTS] [--study STUDY] - [--port PORT] [--host HOST] [--user USER] [--truerun] - [--run_constraints RUN_CONSTRAINTS] [--yes] [--password] - [--list_parameters] [--BDconf BDCONF] - [--binary_operator BINARY_OPERATOR] - -BlackDynamite option parser - -optional arguments: - -h, --help show this help message and exit - -General: - --yes Answer all questions to yes. (default: False) - --binary_operator BINARY_OPERATOR - Set the default binary operator to make requests to - database (default: and) - -BDParser: - --job_constraints JOB_CONSTRAINTS - This allows to constraint run selections by job - properties (default: None) - --study STUDY Specify the study from the BlackDynamite database. - This refers to the schemas in PostgreSQL language - (default: None) - --port PORT Specify data base server port (default: None) - --host HOST Specify data base server address (default: None) - --user USER Specify user name to connect to data base server - (default: None) - --truerun Set this flag if you want to truly perform the action - on base. If not set all action are mainly dryrun - (default: False) - --run_constraints RUN_CONSTRAINTS - This allows to constraint run selections by run - properties (default: None) - --password Flag to request prompt for typing password (default: - False) - --list_parameters Request to list the possible job/run parameters - (default: False) - --BDconf BDCONF Path to a BlackDynamite file (*.bd) configuring - current optons (default: None) - -\end{command} - -An important point is that most of the actions are only applied -when the 'truerun' flag is set. Also, you always have to mention the -host and the study you are working on (all scripts can apply to several studies). -To launch the script and create the database you should launch: - -\begin{command} -./createDB.py --host lsmssrv1.epfl.ch --study MysuperCoolStudy --truerun -\end{command} - - -\section{Creating the jobs} -The goal of the parametric study is to explore a subpart -of the parametric space. We need to create jobs that are -the points to explore. This script is usually named \underline{\code{createJobs.py}}. - -We need to write a python script to generate this set of jobs. -We start by setting the modules and the parser -as for the 'createDB.py' script. -Then we need to create job object: -\begin{command} -job = job.Job(base) -\end{command} - -It is up to us to decide the values to explore. -for convenience, it is possible to insert ranges -of values: - -\begin{command} -job["param1"] = 10 -job["param2"] = [3.14,1.,2.] -job["param3"] = 'toto' -\end{command} - -This will create 3 jobs since we provided a range -of values for the second parameter. -The actual creation is made by calling: - -\begin{command} -base.createParameterSpace(job) -\end{command} - -Launching the script is made with: - -\begin{command} -./createJobs.py --host lsmssrv1.epfl.ch --study test --truerun -\end{command} - -\section{Creating the runs and launching them} - -At this point the jobs are in the database. You need to create runs -that will precise the conditions of the realisation of the jobs. -For example the machine onto -which the job will run, path dependent information, executable information and others. -We have to write the last script, usually named \underline{\code{createRuns.py}} to -specify run creations. - -Again we start with the modules. However this time, we can -use another parser class more adapted to the manipulation of runs: - -\begin{command} -parser = BD.RunParser() -params = parser.parseBDParameters() -base = BD.Base(**params) -\end{command} - -The default parameters for runs will then be automatically -included in the parameters. - -\begin{command} -myrun = run.Run(base) -\end{command} - -Some of the standard parameters might have been parsed directly -by the RunParser, so that we have to forward them to the Run object: - -\begin{command} -myrun.setEntries(params) -\end{command} - -A run now specify what action to perform to realize the job. -Usually, an end-user has a script(s) and wish to attach it to the -run. To attach a file you can for instance do: - -\begin{command} -myrun.addConfigFiles(['file1','file2','launch.sh']) -\end{command} - -Then, one has to specify which of these files is the entry point: - -\begin{command} -myrun.setExecFile("launch.sh") -\end{command} - -Finally, we have to create Run objects and attach them to jobs. -The very first task is to claim the jobs from the database. - -To that end the object JobSelector shall be your friend: -\begin{command} -jobSelector = BD.JobSelector(base) -job_list = jobSelector.selectJobs() -\end{command} - -This will return a job list that you can loop through and -attach the runs to: - -\begin{command} -for j in job_list: - myrun['compiler'] = 'gcc' - myrun.attachToJob(j) -\end{command} - -Everything should then be committed to the database: - -\begin{command} - if params["truerun"] is True: base.commit() -\end{command} - -To create the run one should eventually launch the script by typing: -\begin{command} -./createRuns.py --host lsmssrv1.epfl.ch --study test --machine_name lsmspc41 --run_name toto --nproc int --truerun -\end{command} - -The runs are eventually launched using the tool 'launchRuns.py'. -\begin{command} -./launchRuns.py --host lsmssrv1.epfl.ch --study test --outpath /home/user/ --truerun (--nruns int) -\end{command} - -\section{Accessing and manipulating the database} -The runs can actually be controlled in the database with the tool 'getRunInfo.py', and one can go to the run folder with 'enterRun.py'. -The runs are then launched using the tool 'launchRuns.py'. -\begin{command} -./getRunInfo.py --host lsmssrv1.epfl.ch --study test -./enterRun.py --host lsmssrv1.epfl.ch --study test --run_id ID -\end{command} - -The status of the run can be manually modified using the command 'cleanRuns.py', the default status is CREATED (it can be turned to delete) -\begin{command} -./cleanRuns.py --host lsmssrv1.epfl.ch --study test (--runid ID) --truerun (--delete) -\end{command} -The status and the other run parameters (e.g. the compiler in the example file) can also be modified with 'updateRuns.py'. This can be done in the executed scrip to automatically set the selected parameter -\begin{command} -updateRuns.py --host lsmssrv1.epfl.ch --study test --updates 'state = toto' --truerun -\end{command} - -The function 'canYouDigIt.py' is an example of how to collect data in the runs to draw graphs. Example to plot the crack length in function of the time for different sigma\_c (the study parameter): -\begin{command} -canYouDigIt.py --host lsmssrv1.epfl.ch --study test --quantity time, crack\_length --using \percent 0.x:\percent 1.y --xlabel 'time' --ylabel \percent'crack\_length' --legend 'sigma\_c = \percent j.sigma\_c' -\end{command} - -Eventually, the database can be saved in .zip format to be exported and used offline with 'saveBDStudy.py'. - -\chapter{Instrumenting a simulation code} - -Within you program you need a pusher correctly initialized -in order to push data to the database. The \code{test\_blackdynamite.cc} is an example of such pusher.\\ - -First \blackdynamite includes are required: -\begin{cpp} - #include "blackdynamite.hh" -\end{cpp} -Then you need to create a Pusher object and initialize it. -\begin{cpp} - BlackDynamite::RunManager bd; - bd.startRun(); -\end{cpp} -The constructor by default reads environment variables to get -the database connection and schema informations: -\begin{itemize} -\item RUN\_ID: the identifier of the runid -\item SCHEMA: the schema where the parametric study is to be stored -\item HOST: the database hostname -\end{itemize} - -Then in the places where values are created -you push the values to the database -\begin{cpp} - bd.push(val1,"quantity1",step); - bd.push(val2,"quantity2",step); -\end{cpp} -Step is a stage identifier. It can be the step index within an -explicit loop, or a within a convergence descent or whatever you whish. -It will serve later to compare quantity entries. - -Finally, when the job ended the following call inform the database that the run is finished: -\begin{cpp} - bd.endRun(); -\end{cpp} - -\chapter{Instrumenting a python script} -Within your program you need a run object to push data to the database. This is done by selecting the run from the \code{run\_id} (usually passed as parameter). - -\begin{command} - params = parser.parseBDParameters() - # params['run_id'] should exist - mybase = BD.Base(**params) - runSelector = BD.RunSelector(mybase) - myrun = runSelector(params) -\end{command} - -In order to have time entries for run times, the \code{start()} and \code{finish()} of the run need to be called. - -\begin{command} - myrun.start() - # ... - # Important stuff - # ... - myrun.finish() -\end{command} - -Pushing data is can be done with \code{pushVectorQuantity()} and \code{pushScalarQuantity()}. - -\begin{command} - myrun.pushVectorQuantity(vector_quantity, step, "quantity_id", is_integer) - myrun.pushScalarQuantity(scalar_quantity, step, "quantity_id", is_integer) -\end{command} - -\chapter{Useful Postgresql commands} - -How to list the available schemas ? -\begin{command} -> \dn -\end{command} - -How to get into the schema or the study ? -\begin{command} -> set search path to schema_name; -> set search path to study_name; -\end{command} - - -How to list all the tables ? -\begin{command} -> \d -\end{command} - -How to list entries from a table (like the jobs) ? -\begin{command} -> SELECT * from table_name ; -\end{command} - - - -\end{document} diff --git a/docker/Dockerfile b/docker/Dockerfile new file mode 100644 index 0000000..84899b4 --- /dev/null +++ b/docker/Dockerfile @@ -0,0 +1,29 @@ +FROM debian:buster +MAINTAINER Guillaume Anciaux + +RUN apt-get update && apt-get dist-upgrade -y && apt-get install -y postgresql && apt-get clean -y && rm -r /var/lib/apt/lists/* + +# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed`` +USER postgres + +RUN /etc/init.d/postgresql start &&\ + psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" &&\ + createdb -O docker docker &&\ + psql --command "CREATE USER blackdynamite WITH PASSWORD '';" &&\ + createdb -O blackdynamite blackdynamite + +RUN /etc/init.d/postgresql stop + +# Adjust PostgreSQL configuration so that remote connections to the +# database are possible. +RUN find /etc/postgresql +RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/11/main/pg_hba.conf + +# And add ``listen_addresses`` to ``/etc/postgresql/11/main/postgresql.conf`` +RUN echo "listen_addresses='*'" >> /etc/postgresql/11/main/postgresql.conf + +# Expose the PostgreSQL port +EXPOSE 5432:5432 + +# Set the default command to run when starting the container +CMD ["/usr/lib/postgresql/11/bin/postgres", "-D", "/var/lib/postgresql/11/main", "-c", "config_file=/etc/postgresql/11/main/postgresql.conf"] diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml new file mode 100644 index 0000000..906d8d1 --- /dev/null +++ b/docker/docker-compose.yml @@ -0,0 +1,13 @@ +version: '3.3' + +services: + + blackdynamite: + build: "." + volumes: + - blackdynamitedata:/var/lib/postgresql/ + restart: always + ports: + - "5432:5432" +volumes: + blackdynamitedata: diff --git a/scripts/createUser.py b/scripts/createUser.py index b097885..295b151 100755 --- a/scripts/createUser.py +++ b/scripts/createUser.py @@ -1,72 +1,90 @@ #!/usr/bin/python3 ################################################################ # import getpass import os import stat import psycopg2 import string import argparse +import getpass +import os ################################################################ from random import randint, choice from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT ################################################################ def generatePassword(): characters = string.ascii_letters + string.digits password = "".join(choice(characters) for x in range(randint(8, 16))) return password ################################################################ -def createUser(new_user): +def createUser(user, host): connection_params = dict() - connection_params["user"] = 'postgres' + connection_params["user"] = user + + if host is not None: + connection_params["host"] = host + + if host is None: + host = 'localhost' + connection_params["password"] = getpass.getpass( + '{0}@{1} password: '.format(connection_params['user'], host)) try: connection = psycopg2.connect(**connection_params) except Exception as e: raise Exception(str(e)+'\n'+'*'*30 + '\ncannot connect to database\n' + '*'*30) + new_user = input('new login: ') connection.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT) curs = connection.cursor() try: curs.execute('create user {0}'.format(new_user)) + except Exception as e: - print e - try: - curs.execute('create database {0}'.format(new_user)) - except Exception as e: - print e - try: - curs.execute('alter database {0} owner to {0}'.format(new_user)) - except Exception as e: - print e + print(e) + + print('Setting new password') + curs.execute('grant create on database blackdynamite to {0}'.format( + new_user)) - curs.execute('grant create on database {0} to {0}'.format(new_user)) password = generatePassword() curs.execute('alter role {0} with password \'{1}\' '.format( new_user, password)) fname = '{0}.bd'.format(new_user) + print('Saving information to {0}'.format(fname)) + + try: + os.remove(fname) + except Exception: + pass bdconf = open(fname, 'w') - bdconf.write('password = {0}'.format(password)) + bdconf.write('password = {0}\n'.format(password)) + bdconf.write('host = {0}'.format(host)) bdconf.close() os.chmod(fname, stat.S_IREAD) ################################################################ parser = argparse.ArgumentParser( description='User creation tool for blackdynamite') parser.add_argument("--user", type=str, help="name of the user to create", required=True) +parser.add_argument("--host", type=str, + help="host to connect where to create the user", + default=None) args = parser.parse_args() args = vars(args) new_user = args['user'] -createUser(new_user) +host = args['host'] +createUser(new_user, host)