diff --git a/INSTALL b/INSTALL
index a17d968f8..dfca3ad4a 100644
--- a/INSTALL
+++ b/INSTALL
@@ -1,886 +1,886 @@
-Invenio INSTALLATION
====================
About
=====
This document specifies how to build, customize, and install Invenio
v1.2.1 for the first time. See RELEASE-NOTES if you are upgrading
from a previous Invenio release.
Contents
========
1. Using kickstart scripts
2. Prerequisites
3. Quick instructions for the impatient Invenio admin
4. Detailed instructions for the patient Invenio admin
1. Using kickstart scripts
==========================
You can install Invenio on a Unix-based operating system in an unassisted
automated way by using kickstart scripts. Tested on Ubuntu 12.04, CentOS 6.
The other operating systems will work similarly. Please run:
$ $EDITOR .inveniorc # edit your options
$ source .inveniorc
$ ./scripts/provision-web.sh
$ ./scripts/provision-mysql.sh
$ ./scripts/create-instance.sh
$ ./scripts/populate-instance.sh
$ firefox http://192.168.50.10/record/1
Note that you can use Vagrant to run this in a virtual machine; please see the
.inveniorc file for more details.
You can also read the detailed installation instructions in the sections
below. They basically explain in detail what the kickstart scripts do.
2. Prerequisites
================
Here is the software you need to have around before you
start installing Invenio:
a) Unix-like operating system. The main development and
production platforms for Invenio at CERN are GNU/Linux
distributions Debian, Gentoo, Scientific Linux (aka RHEL),
Ubuntu, but we also develop on Mac OS X. Basically any Unix
system supporting the software listed below should do.
If you are using Debian GNU/Linux ``Lenny'' or later, then you
can install most of the below-mentioned prerequisites and
recommendations by running:
$ sudo aptitude install python-dev apache2-mpm-prefork \
mysql-server mysql-client python-mysqldb \
python-4suite-xml python-simplejson python-xml \
gnuplot poppler-utils \
gs-common clisp gettext libapache2-mod-wsgi unzip \
python-dateutil python-rdflib python-pyparsing \
python-gnuplot python-magic pdftk html2text giflib-tools \
pstotext netpbm python-pypdf python-chardet python-lxml \
python-unidecode redis-server python-redis
You may also want to install some of the following packages,
if you have them available on your concrete architecture:
$ sudo aptitude install sbcl cmucl pylint pychecker pyflakes \
python-profiler python-epydoc libapache2-mod-xsendfile \
openoffice.org python-utidylib python-beautifulsoup \
libhdf5-dev
(Note that if you use pip to manage your Python dependencies
instead of operating system packages, please see the section
(d) below on how to use pip instead of aptitude.)
Moreover, you should install some Message Transfer Agent (MTA)
such as Postfix so that Invenio can email notification
alerts or registration information to the end users, contact
moderators and reviewers of submitted documents, inform
administrators about various runtime system information, etc:
$ sudo aptitude install postfix
After running the above-quoted aptitude command(s), you can
proceed to configuring your MySQL server instance
(max_allowed_packet in my.cnf, see item b below) and then to
installing the Invenio software package in the section 1
below.
If you are using another operating system, then please
continue reading the rest of this prerequisites section, and
please consult our wiki pages for any concrete hints for your
specific operating system.
<https://twiki.cern.ch/twiki/bin/view/CDS/Invenio>
b) MySQL server (may be on a remote machine), and MySQL client
(must be available locally too). MySQL versions 4.1 or 5.0
are supported. Please set the variable "max_allowed_packet"
in your "my.cnf" init file to at least 4M. (For sites such as
INSPIRE, having 1M records with 10M citer-citee pairs in its
citation map, you may need to increase max_allowed_packet to
1G.) You may perhaps also want to run your MySQL server
natively in UTF-8 mode by setting "default-character-set=utf8"
in various parts of your "my.cnf" file, such as in the
"[mysql]" part and elsewhere; but this is not really required.
Note also that you may encounter problems when MySQL is run in
"strict mode"; you may want to configure your "my.cnf" in order
to avoid using strict mode (such as `STRICT_ALL_TABLES`).
<http://mysql.com/>
c) Redis server (may be on a remote machine) for user session
management and caching purposes. By default, Invenio would
use Redis to store sessions, so it is highly recommended to
install it. However, if you do not want to use Redis, you can
change CFG_WEBSESSION_STORAGE settings in invenio-local.conf
and MySQL will be used for session management instead.
<http://redis.io/>
d) Apache 2 server, with support for loading DSO modules, and
optionally with SSL support for HTTPS-secure user
authentication, and mod_xsendfile for off-loading file
downloads away from Invenio processes to Apache.
<http://httpd.apache.org/>
<http://tn123.ath.cx/mod_xsendfile/>
e) Python v2.6 or above:
<http://python.org/>
as well as the following Python modules:
- (mandatory) MySQLdb (version >= 1.2.1_p2; see below)
<http://sourceforge.net/projects/mysql-python>
- (mandatory) Pyparsing, for document parsing
<http://pyparsing.wikispaces.com/>
- (mandatory) unidecode, for ASCII representation of Unicode text:
<https://pypi.python.org/pypi/Unidecode>
- (recommended) Redis connector:
<https://pypi.python.org/pypi/redis/>
- (recommended) Nydus, Redis consistent hashing connector:
<https://github.com/disqus/nydus>
- (recommended) python-dateutil, for complex date processing:
<http://labix.org/python-dateutil>
- (recommended) PyRXP, for very fast XML MARC processing:
<http://www.reportlab.org/pyrxp.html>
- (recommended) lxml, for XML/XLST processing:
<http://lxml.de/>
- (recommended) Gnuplot.Py, for producing graphs:
<http://gnuplot-py.sourceforge.net/>
- (recommended) Snowball Stemmer, for stemming:
<http://snowball.tartarus.org/wrappers/PyStemmer-1.0.1.tar.gz>
- (recommended) py-editdist, for record merging:
<http://www.mindrot.org/projects/py-editdist/>
- (recommended) numpy, for citerank methods:
<http://numpy.scipy.org/>
- (recommended) magic, for full-text file handling:
<http://www.darwinsys.com/file/>
- (recommended) cerberus, extensible validation for Python dictionaries.
<http://cerberus.readthedocs.org/>
- (optional) libxml2-python, for XML/XLST processing:
<ftp://xmlsoft.org/libxml2/python/>
- (optional) chardet, for character encoding detection:
<http://chardet.feedparser.org/>
- (optional) 4suite, slower alternative to PyRXP and
libxml2-python:
<http://4suite.org/>
- (optional) feedparser, for web journal creation:
<http://feedparser.org/>
- (optional) RDFLib, to use RDF ontologies and thesauri:
<http://rdflib.net/>
- (optional) mechanize, to run regression web test suite:
<http://wwwsearch.sourceforge.net/mechanize/>
- (optional) python-mock, mocking library for the test suite:
<http://www.voidspace.org.uk/python/mock/>
- (optional) utidylib, for HTML washing:
<http://utidylib.berlios.de/>
- (optional) Beautiful Soup, for HTML washing:
<http://www.crummy.com/software/BeautifulSoup/>
- (optional) Python Twitter (and its dependencies) if you want
to use the Twitter Fetcher bibtasklet:
<http://code.google.com/p/python-twitter/>
- (optional) Python OpenID if you want to enable OpenID support
for authentication:
<http://pypi.python.org/pypi/python-openid/>
- (optional) Python Rauth if you want to enable OAuth 1.0/2.0
support for authentication (depends on Python-2.6 or later):
<http://packages.python.org/rauth/>
- (optional) libhdf5-7, libhdf5-dev, python-h5py, in order to
run author disambiguation.
<https://code.google.com/p/h5py/>
Note that if you are using pip to install and manage your
Python dependencies, then you can run:
$ sudo pip install -r requirements.txt
$ sudo pip install -r requirements-extras.txt
to install all manadatory, recommended, and optional packages
mentioned above.
f) mod_wsgi Apache module. Versions 3.x and above are
recommended.
<http://code.google.com/p/modwsgi/>
g) If you want to be able to extract references from PDF fulltext
files, then you need to install pdftotext version 3 at least.
<http://poppler.freedesktop.org/>
<http://www.foolabs.com/xpdf/home.html>
h) If you want to be able to search for words in the fulltext
files (i.e. to have fulltext indexing) or to stamp submitted
files, then you need as well to install some of the following
tools:
- for Microsoft Office/OpenOffice.org document conversion:
OpenOffice.org
<http://www.openoffice.org/>
- for PDF file stamping: pdftk, pdf2ps
<http://www.accesspdf.com/pdftk/>
<http://www.cs.wisc.edu/~ghost/doc/AFPL/>
- for PDF files: pdftotext or pstotext
<http://poppler.freedesktop.org/>
<http://www.foolabs.com/xpdf/home.html>
<http://www.cs.wisc.edu/~ghost/doc/AFPL/>
- for PostScript files: pstotext or ps2ascii
<http://www.cs.wisc.edu/~ghost/doc/AFPL/>
- for DjVu creation, elaboration: DjVuLibre
<http://djvu.sourceforge.net>
- to perform OCR: OCRopus (tested only with release 0.3.1)
<http://code.google.com/p/ocropus/>
- to perform different image elaborations: ImageMagick
<http://www.imagemagick.org/>
- to generate PDF after OCR: netpbm, ReportLab and pyPdf or pyPdf2
<http://netpbm.sourceforge.net/>
<http://www.reportlab.org/rl_toolkit.html>
<http://pybrary.net/pyPdf/>
<http://knowah.github.io/PyPDF2/>
i) If you have chosen to install fast XML MARC Python processors
in the step d) above, then you have to install the parsers
themselves:
- (optional) 4suite:
<http://4suite.org/>
j) (recommended) Gnuplot, the command-line driven interactive
plotting program. It is used to display download and citation
history graphs on the Detailed record pages on the web
interface. Note that Gnuplot must be compiled with PNG output
support, that is, with the GD library. Note also that Gnuplot
is not required, only recommended.
<http://www.gnuplot.info/>
k) (recommended) A Common Lisp implementation, such as CLISP,
SBCL or CMUCL. It is used for the web server log analysing
tool and the metadata checking program. Note that any of the
three implementations CLISP, SBCL, or CMUCL will do. CMUCL
produces fastest machine code, but it does not support UTF-8
yet. Pick up CLISP if you don't know what to do. Note that a
Common Lisp implementation is not required, only recommended.
<http://clisp.cons.org/>
<http://www.cons.org/cmucl/>
<http://sbcl.sourceforge.net/>
l) GNU gettext, a set of tools that makes it possible to
translate the application in multiple languages.
<http://www.gnu.org/software/gettext/>
This is available by default on many systems.
m) (recommended) xlwt 0.7.2, Library to create spreadsheet files
compatible with MS Excel 97/2000/XP/2003 XLS files, on any
platform, with Python 2.3 to 2.6
<http://pypi.python.org/pypi/xlwt>
n) (recommended) matplotlib 1.0.0 is a python 2D plotting library
which produces publication quality figures in a variety of
hardcopy formats and interactive environments across
platforms. matplotlib can be used in python scripts, the
python and ipython shell (ala MATLAB® or Mathematica®),
web application servers, and six graphical user interface
toolkits. It is used to generate pie graphs in the custom
summary query (WebStat)
<http://matplotlib.sourceforge.net>
o) (optional) FFmpeg, an open-source tools an libraries collection
to convert video and audio files. It makes use of both internal
as well as external libraries to generate videos for the web, such
as Theora, WebM and H.264 out of almost any thinkable video input.
FFmpeg is needed to run video related modules and submission workflows
in Invenio. The minimal configuration of ffmpeg for the Invenio demo site
requires a number of external libraries. It is highly recommended
to remove all installed versions and packages that are comming with
various Linux distributions and install the latest versions from
sources. Additionally, you will need the Mediainfo Library for multimedia
metadata handling.
Minimum libraries for the demo site:
- the ffmpeg multimedia encoder tools
<http://ffmpeg.org/>
- a library for jpeg images needed for thumbnail extraction
<http://www.openjpeg.org/>
- a library for the ogg container format, needed for Vorbis and Theora
<http://www.xiph.org/ogg/>
- the OGG Vorbis audi codec library
<http://www.vorbis.com/>
- the OGG Theora video codec library
<http://www.theora.org/>
- the WebM video codec library
<http://www.webmproject.org/>
- the mediainfo library for multimedia metadata
<http://mediainfo.sourceforge.net/>
Recommended for H.264 video (!be aware of licensing issues!):
- a library for H.264 video encoding
<http://www.videolan.org/developers/x264.html>
- a library for Advanced Audi Coding
<http://www.audiocoding.com/faac.html>
- a library for MP3 encoding
<http://lame.sourceforge.net/>
Note that the configure script checks whether you have all the
prerequisite software installed and that it won't let you continue
unless everything is in order. It also warns you if it cannot find
some optional but recommended software.
3. Quick instructions for the impatient Invenio admin
=====================================================
3a. Installation
----------------
$ cd $HOME/src/
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz.md5
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz.sig
$ md5sum -c invenio-1.2.1.tar.gz.md5
$ gpg --verify invenio-1.2.1.tar.gz.sig invenio-1.2.1.tar.gz
$ tar xvfz invenio-1.2.1.tar.gz
$ cd invenio-1.2.1
$ ./configure
$ make
$ make install
$ make install-mathjax-plugin ## optional
$ make install-jquery-plugins ## optional
$ make install-ckeditor-plugin ## optional
$ make install-pdfa-helper-files ## optional
$ make install-mediaelement ## optional
$ make install-solrutils ## optional
$ make install-js-test-driver ## optional
3b. Configuration
-----------------
$ sudo chown -R www-data.www-data /opt/invenio
$ sudo -u www-data $EDITOR /opt/invenio/etc/invenio-local.conf
$ sudo -u www-data /opt/invenio/bin/inveniocfg --update-all
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-tables
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-bibfield-conf
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-webstat-conf
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-apache-conf
- $ sudo /etc/init.d/apache2 restart
+ $ sudo /usr/sbin/service apache2 restart
$ sudo -u www-data /opt/invenio/bin/inveniocfg --check-openoffice
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-demo-site
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-demo-records
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-unit-tests
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-regression-tests
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-web-tests
$ sudo -u www-data /opt/invenio/bin/inveniocfg --remove-demo-records
$ sudo -u www-data /opt/invenio/bin/inveniocfg --drop-demo-site
$ firefox http://your.site.com/help/admin/howto-run
4. Detailed instructions for the patient Invenio admin
======================================================
4a. Installation
----------------
The Invenio uses standard GNU autoconf method to build and
install its files. This means that you proceed as follows:
$ cd $HOME/src/
Change to a directory where we will build the Invenio
sources. (The built files will be installed into different
"target" directories later.)
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz.md5
$ wget http://invenio-software.org/download/invenio-1.2.1.tar.gz.sig
Fetch Invenio source tarball from the distribution server,
together with MD5 checksum and GnuPG cryptographic signature
files useful for verifying the integrity of the tarball.
$ md5sum -c invenio-1.2.1.tar.gz.md5
Verify MD5 checksum.
$ gpg --verify invenio-1.2.1.tar.gz.sig invenio-1.2.1.tar.gz
Verify GnuPG cryptographic signature. Note that you may
first have to import my public key into your keyring, if you
haven't done that already:
$ gpg --keyserver pool.sks-keyservers.net --recv-key 0xBA5A2B67
The output of the gpg --verify command should then read:
Good signature from "Tibor Simko <tibor@simko.info>"
You can safely ignore any trusted signature certification
warning that may follow after the signature has been
successfully verified.
$ tar xvfz invenio-1.2.1.tar.gz
Untar the distribution tarball.
$ cd invenio-1.2.1
Go to the source directory.
$ ./configure
Configure Invenio software for building on this specific
platform. You can use the following optional parameters:
--prefix=/opt/invenio
Optionally, specify the Invenio general
installation directory (default is /opt/invenio).
It will contain command-line binaries and program
libraries containing the core Invenio
functionality, but also store web pages, runtime log
and cache information, document data files, etc.
Several subdirs like `bin', `etc', `lib', or `var'
will be created inside the prefix directory to this
effect. Note that the prefix directory should be
chosen outside of the Apache htdocs tree, since only
one its subdirectory (prefix/var/www) is to be
accessible directly via the Web (see below).
Note that Invenio won't install to any other
directory but to the prefix mentioned in this
configuration line.
--with-python=/opt/python/bin/python2.7
Optionally, specify a path to some specific Python
binary. This is useful if you have more than one
Python installation on your system. If you don't set
this option, then the first Python that will be found
in your PATH will be chosen for running Invenio.
--with-mysql=/opt/mysql/bin/mysql
Optionally, specify a path to some specific MySQL
client binary. This is useful if you have more than
one MySQL installation on your system. If you don't
set this option, then the first MySQL client
executable that will be found in your PATH will be
chosen for running Invenio.
--with-clisp=/opt/clisp/bin/clisp
Optionally, specify a path to CLISP executable. This
is useful if you have more than one CLISP
installation on your system. If you don't set this
option, then the first executable that will be found
in your PATH will be chosen for running Invenio.
--with-cmucl=/opt/cmucl/bin/lisp
Optionally, specify a path to CMUCL executable. This
is useful if you have more than one CMUCL
installation on your system. If you don't set this
option, then the first executable that will be found
in your PATH will be chosen for running Invenio.
--with-sbcl=/opt/sbcl/bin/sbcl
Optionally, specify a path to SBCL executable. This
is useful if you have more than one SBCL
installation on your system. If you don't set this
option, then the first executable that will be found
in your PATH will be chosen for running Invenio.
--with-openoffice-python
Optionally, specify the path to the Python interpreter
embedded with OpenOffice.org. This is normally not
contained in the normal path. If you don't specify this
it won't be possible to use OpenOffice.org to convert from and
to Microsoft Office and OpenOffice.org documents.
This configuration step is mandatory. Usually, you do this
step only once.
(Note that if you are building Invenio not from a
released tarball, but from the Git sources, then you have to
generate the configure file via autotools:
$ sudo aptitude install automake1.9 autoconf
$ aclocal-1.9
$ automake-1.9 -a
$ autoconf
after which you proceed with the usual configure command.)
$ make
Launch the Invenio build. Since many messages are printed
during the build process, you may want to run it in a
fast-scrolling terminal such as rxvt or in a detached screen
session.
During this step all the pages and scripts will be
pre-created and customized based on the config you have
edited in the previous step.
Note that on systems such as FreeBSD or Mac OS X you have to
use GNU make ("gmake") instead of "make".
$ make install
Install the web pages, scripts, utilities and everything
needed for Invenio runtime into respective installation
directories, as specified earlier by the configure command.
Note that if you are installing Invenio for the first
time, you will be asked to create symbolic link(s) from
Python's site-packages system-wide directory(ies) to the
installation location. This is in order to instruct Python
where to find Invenio's Python files. You will be
hinted as to the exact command to use based on the
parameters you have used in the configure command.
$ make install-mathjax-plugin ## optional
This will automatically download and install in the proper
place MathJax, a JavaScript library to render LaTeX formulas
in the client browser.
Note that in order to enable the rendering you will have to
set the variable CFG_WEBSEARCH_USE_MATHJAX_FOR_FORMATS in
invenio-local.conf to a suitable list of output format
codes. For example:
CFG_WEBSEARCH_USE_MATHJAX_FOR_FORMATS = hd,hb
$ make install-jquery-plugins ## optional
This will automatically download and install in the proper
place jQuery and related plugins. They are used for AJAX
applications such as the record editor.
Note that `unzip' is needed when installing jquery plugins.
$ make install-ckeditor-plugin ## optional
This will automatically download and install in the proper
place CKeditor, a WYSIWYG Javascript-based editor (e.g. for
the WebComment module).
Note that in order to enable the editor you have to set the
CFG_WEBCOMMENT_USE_RICH_EDITOR to True.
$ make install-pdfa-helper-files ## optional
This will automatically download and install in the proper
place the helper files needed to create PDF/A files out of
existing PDF files.
$ make install-mediaelement ## optional
This will automatically download and install the MediaElementJS
HTML5 video player that is needed for videos on the DEMO site.
$ make install-solrutils ## optional
This will automatically download and install a Solr instance
which can be used for full-text searching. See CFG_SOLR_URL
variable in the invenio.conf. Note that the admin later has
to take care of running init.d scripts which would start the
Solr instance automatically.
$ make install-js-test-driver ## optional
This will automatically download and install JsTestDriver
which is needed to run JS unit tests. Recommended for developers.
4b. Configuration
-----------------
Once the basic software installation is done, we proceed to
configuring your Invenio system.
$ sudo chown -R www-data.www-data /opt/invenio
For the sake of simplicity, let us assume that your Invenio
installation will run under the `www-data' user process
identity. The above command changes ownership of installed
files to www-data, so that we shall run everything under
this user identity from now on.
For production purposes, you would typically enable Apache
server to read all files from the installation place but to
write only to the `var' subdirectory of your installation
place. You could achieve this by configuring Unix directory
group permissions, for example.
$ sudo -u www-data $EDITOR /opt/invenio/etc/invenio-local.conf
Customize your Invenio installation. Please read the
'invenio.conf' file located in the same directory that
contains the vanilla default configuration parameters of
your Invenio installation. If you want to customize some of
these parameters, you should create a file named
'invenio-local.conf' in the same directory where
'invenio.conf' lives and you should write there only the
customizations that you want to be different from the
vanilla defaults.
Here is a realistic, minimalist, yet production-ready
example of what you would typically put there:
$ cat /opt/invenio/etc/invenio-local.conf
[Invenio]
CFG_SITE_NAME = John Doe's Document Server
CFG_SITE_NAME_INTL_fr = Serveur des Documents de John Doe
CFG_SITE_URL = http://your.site.com
CFG_SITE_SECURE_URL = https://your.site.com
CFG_SITE_ADMIN_EMAIL = john.doe@your.site.com
CFG_SITE_SUPPORT_EMAIL = john.doe@your.site.com
CFG_WEBALERT_ALERT_ENGINE_EMAIL = john.doe@your.site.com
CFG_WEBCOMMENT_ALERT_ENGINE_EMAIL = john.doe@your.site.com
CFG_WEBCOMMENT_DEFAULT_MODERATOR = john.doe@your.site.com
CFG_BIBAUTHORID_AUTHOR_TICKET_ADMIN_EMAIL = john.doe@your.site.com
CFG_BIBCATALOG_SYSTEM_EMAIL_ADDRESS = john.doe@your.site.com
CFG_DATABASE_HOST = localhost
CFG_DATABASE_NAME = invenio
CFG_DATABASE_USER = invenio
CFG_DATABASE_PASS = my123p$ss
CFG_BIBDOCFILE_ENABLE_BIBDOCFSINFO_CACHE = 1
You should override at least the parameters mentioned above
in order to define some very essential runtime parameters
such as the name of your document server (CFG_SITE_NAME and
CFG_SITE_NAME_INTL_*), the visible URL of your document
server (CFG_SITE_URL and CFG_SITE_SECURE_URL), the email
address of the local Invenio administrator, comment
moderator, and alert engine (CFG_SITE_SUPPORT_EMAIL,
CFG_SITE_ADMIN_EMAIL, etc), and last but not least your
database credentials (CFG_DATABASE_*).
If this is a first installation of Invenio it is recommended
you set the CFG_BIBDOCFILE_ENABLE_BIBDOCFSINFO_CACHE
variable to 1. If this is instead an upgrade from an existing
installation don't add it until you have run:
$ bibdocfile --fix-bibdocfsinfo-cache .
The Invenio system will then read both the default
invenio.conf file and your customized invenio-local.conf
file and it will override any default options with the ones
you have specifield in your local file. This cascading of
configuration parameters will ease your future upgrades.
If you want to have multiple Invenio instances for distributed
video encoding, you need to share the same configuration amongs
them and make some of the folders of the Invenio installation
available for all nodes.
Configure the allowed tasks for every node:
CFG_BIBSCHED_NODE_TASKS = {
"hostname_machine1" : ["bibindex", "bibupload",
"bibreformat","webcoll", "bibtaskex", "bibrank",
"oaiharvest", "oairepositoryupdater", "inveniogc",
"webstatadmin", "bibclassify", "bibexport",
"dbdump", "batchuploader", "bibauthorid", "bibtasklet"],
"hostname_machine2" : ['bibencode',]
}
Share the following directories among Invenio instances:
/var/tmp-shared
hosts video uploads in a temporary form
/var/tmp-shared/bibencode/jobs
hosts new job files for the video encoding daemon
/var/tmp-shared/bibencode/jobs/done
hosts job files that have been processed by the daemon
/var/data/files
hosts fulltext and media files associated to records
/var/data/submit
hosts files created during submissions
$ sudo -u www-data /opt/invenio/bin/inveniocfg --update-all
Make the rest of the Invenio system aware of your
invenio-local.conf changes. This step is mandatory each
time you edit your conf files.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-tables
If you are installing Invenio for the first time, you
have to create database tables.
Note that this step checks for potential problems such as
the database connection rights and may ask you to perform
some more administrative steps in case it detects a problem.
Notably, it may ask you to set up database access
permissions, based on your configure values.
If you are installing Invenio for the first time, you
have to create a dedicated database on your MySQL server
that the Invenio can use for its purposes. Please
contact your MySQL administrator and ask him to execute the
commands this step proposes you.
At this point you should now have successfully completed the
"make install" process. We continue by setting up the
Apache web server.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-bibfield-conf
Load the configuration file of the BibField module. It will
create `bibfield_config.py' file. (FIXME: When BibField
becomes essential part of Invenio, this step should be later
automatised so that people do not have to run it manually.)
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-webstat-conf
Load the configuration file of webstat module. It will create
the tables in the database for register customevents, such as
basket hits.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-apache-conf
Running this command will generate Apache virtual host
configurations matching your installation. You will be
instructed to check created files (usually they are located
under /opt/invenio/etc/apache/) and edit your httpd.conf
to activate Invenio virtual hosts.
If you are using Debian GNU/Linux ``Lenny'' or later, then
you can do the following to create your SSL certificate and
to activate your Invenio vhosts:
## make SSL certificate:
$ sudo aptitude install ssl-cert
$ sudo mkdir /etc/apache2/ssl
$ sudo /usr/sbin/make-ssl-cert /usr/share/ssl-cert/ssleay.cnf \
/etc/apache2/ssl/apache.pem
## add Invenio web sites:
$ sudo ln -s /opt/invenio/etc/apache/invenio-apache-vhost.conf \
/etc/apache2/sites-available/invenio
$ sudo ln -s /opt/invenio/etc/apache/invenio-apache-vhost-ssl.conf \
/etc/apache2/sites-available/invenio-ssl
## disable Debian's default web site:
$ sudo /usr/sbin/a2dissite default
## enable Invenio web sites:
$ sudo /usr/sbin/a2ensite invenio
$ sudo /usr/sbin/a2ensite invenio-ssl
## enable SSL module:
$ sudo /usr/sbin/a2enmod ssl
## if you are using xsendfile module, enable it too:
$ sudo /usr/sbin/a2enmod xsendfile
If you are using another operating system, you should do the
equivalent, for example edit your system-wide httpd.conf and
put the following include statements:
Include /opt/invenio/etc/apache/invenio-apache-vhost.conf
Include /opt/invenio/etc/apache/invenio-apache-vhost-ssl.conf
Note that you may need to adapt generated vhost file
snippets to match your concrete operating system specifics.
For example, the generated configuration snippet will
preload Invenio WSGI daemon application upon Apache start up
for faster site response. The generated configuration
assumes that you are using mod_wsgi version 3 or later. If
you are using the old legacy mod_wsgi version 2, then you
would need to comment out the WSGIImportScript directive
from the generated snippet, or else move the WSGI daemon
setup to the top level, outside of the VirtualHost section.
Note also that you may want to tweak the generated Apache
vhost snippet for performance reasons, especially with
respect to WSGIDaemonProcess parameters. For example, you
can increase the number of processes from the default value
`processes=5' if you have lots of RAM and if many concurrent
users may access your site in parallel. However, note that
you must use `threads=1' there, because Invenio WSGI daemon
processes are not fully thread safe yet. This may change in
the future.
- $ sudo /etc/init.d/apache2 restart
+ $ sudo /usr/sbin/service apache2 restart
Please ask your webserver administrator to restart the
Apache server after the above "httpd.conf" changes.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --check-openoffice
If you plan to support MS Office or Open Document Format
files in your installation, you should check whether
LibreOffice or OpenOffice.org is well integrated with
Invenio by running the above command. You may be asked to
create a temporary directory for converting office files
with special ownership (typically as user nobody) and
permissions. Note that you can do this step later.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --create-demo-site
This step is recommended to test your local Invenio
installation. It should give you our "Atlantis Institute of
Science" demo installation, exactly as you see it at
<http://demo.invenio-software.org/>.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --load-demo-records
Optionally, load some demo records to be able to test
indexing and searching of your local Invenio demo
installation.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-unit-tests
Optionally, you can run the unit test suite to verify the
unit behaviour of your local Invenio installation. Note
that this command should be run only after you have
installed the whole system via `make install'.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-regression-tests
Optionally, you can run the full regression test suite to
verify the functional behaviour of your local Invenio
installation. Note that this command requires to have
created the demo site and loaded the demo records. Note
also that running the regression test suite may alter the
database content with junk data, so that rebuilding the
demo site is strongly recommended afterwards.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --run-web-tests
Optionally, you can run additional automated web tests
running in a real browser. This requires to have Firefox
with the Selenium IDE extension installed.
<http://en.www.mozilla.com/en/firefox/>
<http://selenium-ide.openqa.org/>
$ sudo -u www-data /opt/invenio/bin/inveniocfg --remove-demo-records
Optionally, remove the demo records loaded in the previous
step, but keeping otherwise the demo collection, submission,
format, and other configurations that you may reuse and
modify for your own production purposes.
$ sudo -u www-data /opt/invenio/bin/inveniocfg --drop-demo-site
Optionally, drop also all the demo configuration so that
you'll end up with a completely blank Invenio system.
However, you may want to find it more practical not to drop
the demo site configuration but to start customizing from
there.
$ firefox http://your.site.com/help/admin/howto-run
In order to start using your Invenio installation, you
can start indexing, formatting and other daemons as
indicated in the "HOWTO Run" guide on the above URL. You
can also use the Admin Area web interfaces to perform
further runtime configurations such as the definition of
data collections, document types, document formats, word
indexes, etc.
$ sudo ln -s /opt/invenio/etc/bash_completion.d/inveniocfg \
/etc/bash_completion.d/inveniocfg
Optionally, if you are using Bash shell completion, then
you may want to create the above symlink in order to
configure completion for the inveniocfg command.
Happy hacking and thanks for flying Invenio.
- Invenio Development Team
Email: info@invenio-software.org
IRC: #invenio on irc.freenode.net
Twitter: http://twitter.com/inveniosoftware
Github: http://github.com/inveniosoftware
URL: http://invenio-software.org
diff --git a/Vagrantfile b/Vagrantfile
index 3d5359b0a..049660b10 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -1,68 +1,71 @@
# -*- coding: utf-8 -*-
#
# This file is part of Invenio.
# Copyright (C) 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# This Vagrant configuration is suitable for Invenio demo site installation as
# governed by `.inveniorc`. It uses separate dedicated VMs for various services
# in order to better emulate production environment conditions. You can install
# an Invenio demo site by running:
#
# $ vagrant up --no-parallel
# $ vagrant ssh web -c 'source .inveniorc && /vagrant/scripts/create-instance.sh'
# $ vagrant ssh web -c 'source .inveniorc && /vagrant/scripts/populate-instance.sh'
# $ firefox http://192.168.50.10/record/1
# $ vagrant ssh web -c 'source .inveniorc && sudo -u www-data /opt/invenio/bin/inveniocfg --run-unit-tests'
# $ vagrant ssh web -c 'source .inveniorc && sudo -u www-data /opt/invenio/bin/inveniocfg --run-regression-tests --yes-i-know'
-#OS = 'hfm4/centos6'
-OS = 'ubuntu/precise64'
+# Tested on:
+#
+# OS = 'hfm4/centos6' # CentOS 6
+# OS = 'ubuntu/precise64' # Ubuntu 12.04 LTS Precise Pangolin -- used by Travis CI
+OS = 'ubuntu/trusty64' # Ubuntu 14.04 LTS Trusty Tahr
Vagrant.configure("2") do |config|
if Vagrant.has_plugin?("vagrant-cachier")
config.cache.scope = :box
end
config.vm.define "web" do |web|
web.vm.box = OS
web.vm.hostname = 'web'
web.vm.provision "file", source: ".inveniorc", destination: ".inveniorc"
web.vm.provision "shell", inline: "source .inveniorc && /vagrant/scripts/provision-web.sh", privileged: false
web.vm.network "forwarded_port", guest: 80, host: 80
web.vm.network "forwarded_port", guest: 443, host: 443
web.vm.network "private_network", ip: ENV.fetch('INVENIO_WEB_HOST','192.168.50.10')
web.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", "4096"]
vb.customize ["modifyvm", :id, "--cpus", 2]
end
end
config.vm.define "mysql" do |mysql|
mysql.vm.box = OS
mysql.vm.hostname = 'mysql'
mysql.vm.provision "file", source: ".inveniorc", destination: ".inveniorc"
mysql.vm.provision "shell", inline: "source .inveniorc && /vagrant/scripts/provision-mysql.sh", privileged: false
mysql.vm.network "private_network", ip: ENV.fetch('INVENIO_MYSQL_HOST','192.168.50.11')
end
end
diff --git a/scripts/create-instance.sh b/scripts/create-instance.sh
index d52334771..c37730084 100755
--- a/scripts/create-instance.sh
+++ b/scripts/create-instance.sh
@@ -1,264 +1,281 @@
#!/usr/bin/env bash
#
# This file is part of Invenio.
# Copyright (C) 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# quit on errors:
set -o errexit
# check environment variables:
if [ "${INVENIO_MYSQL_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_MYSQL_HOST=192.168.50.11"
exit 1
fi
if [ "${INVENIO_MYSQL_DBNAME}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBNAME before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBNAME=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBUSER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBUSER before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBUSER=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBPASS}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBPASS before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBPASS=dbpass123"
exit 1
fi
if [ "${INVENIO_WEB_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_HOST=192.168.50.10"
exit 1
fi
if [ "${INVENIO_WEB_DSTDIR}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_DSTDIR before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_DSTDIR=/opt/invenio"
exit 1
fi
if [ "${INVENIO_WEB_USER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_USER before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_USER=www-data"
exit 1
fi
# check optional environment variables:
INVENIO_WEB_SMTP_PORT=${INVENIO_WEB_SMTP_PORT:=25}
# quit on unbound symbols:
set -o nounset
# runs inside virtual environment?
VIRTUAL_ENV=${VIRTUAL_ENV:=}
# runs as root or needs sudo?
if [[ "$EUID" -ne 0 ]]; then
sudo='sudo'
else
sudo=''
fi
-create_apache_vhost_ubuntu_precise () {
+create_apache_vhost_ubuntu12 () {
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y ssl-cert
sudo mkdir -p /etc/apache2/ssl
if [ ! -e /etc/apache2/ssl/apache.pem ]; then
sudo DEBIAN_FRONTEND=noninteractive /usr/sbin/make-ssl-cert \
/usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem
fi
if [ ! -L /etc/apache2/sites-available/invenio.conf ]; then
sudo ln -fs "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf" \
/etc/apache2/sites-available/invenio.conf
fi
if [ ! -e "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf" ]; then
# create them empty for the time being so that apache would start
sudo mkdir -p "${INVENIO_WEB_DSTDIR}/etc/apache/"
sudo touch "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf"
sudo chown -R "${INVENIO_WEB_USER}.${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}"
fi
if [ ! -L /etc/apache2/sites-available/invenio-ssl.conf ]; then
sudo ln -fs "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf" \
/etc/apache2/sites-available/invenio-ssl.conf
fi
if [ ! -e "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf" ]; then
# create them empty for the time being so that apache would start
sudo mkdir -p "${INVENIO_WEB_DSTDIR}/etc/apache/"
sudo touch "${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf"
sudo chown -R "${INVENIO_WEB_USER}.${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}"
fi
if [ -e /etc/apache2/sites-available/default-ssl ]; then
sudo /usr/sbin/a2dissite "*default*"
fi
sudo /usr/sbin/a2ensite "invenio*"
sudo /usr/sbin/a2enmod ssl
sudo /usr/sbin/a2enmod version || echo "[WARNING] Ignoring 'a2enmod version' command; hoping IfVersion is built-in."
sudo /usr/sbin/a2enmod xsendfile
- sudo /etc/init.d/apache2 restart
+ sudo /usr/sbin/service apache2 restart
+}
+
+create_apache_vhost_ubuntu14 () {
+ create_apache_vhost_ubuntu12
}
create_apache_vhost_centos6 () {
if ! grep -q "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf" /etc/httpd/conf/httpd.conf; then
echo "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf" | sudo tee -a /etc/httpd/conf/httpd.conf
fi
if ! grep -q "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf" /etc/httpd/conf/httpd.conf; then
echo "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf" | sudo tee -a /etc/httpd/conf/httpd.conf
fi
if ! grep -q "TraceEnable off" /etc/httpd/conf/httpd.conf; then
echo "TraceEnable off" | sudo tee -a /etc/httpd/conf/httpd.conf
fi
if ! grep -q "SSLProtocol all -SSLv2" /etc/httpd/conf/httpd.conf; then
echo "SSLProtocol all -SSLv2" | sudo tee -a /etc/httpd/conf/httpd.conf
fi
sudo sed -i 's,^Alias /error/,#Alias /error/,g' /etc/httpd/conf/httpd.conf
}
create_symlinks () {
$sudo mkdir -p "${INVENIO_WEB_DSTDIR}"
$sudo chown "${INVENIO_WEB_USER}.${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}"
$sudo -u "${INVENIO_WEB_USER}" mkdir -p "${INVENIO_WEB_DSTDIR}/lib/python/invenio"
for pythonversion in python2.4 python2.6 python2.7; do
for libversion in lib lib64 local/lib local/lib64; do
for packageversion in site-packages dist-packages; do
if [ -d "/usr/$libversion/$pythonversion/$packageversion/" ] && [ ! -L "/usr/$libversion/$pythonversion/$packageversion/invenio" ]; then
$sudo ln -s "${INVENIO_WEB_DSTDIR}/lib/python/invenio" "/usr/$libversion/$pythonversion/$packageversion/invenio"
fi
done
done
done
}
install_sources () {
cd "${INVENIO_SRCDIR}"
rm -rf autom4te.cache/
aclocal
automake -a
autoconf
./configure --prefix="${INVENIO_WEB_DSTDIR}"
make clean -s
make -s
sudo -u "${INVENIO_WEB_USER}" make -s install
sudo -u "${INVENIO_WEB_USER}" make -s install-jquery-plugins
sudo -u "${INVENIO_WEB_USER}" make -s install-mathjax-plugin
sudo -u "${INVENIO_WEB_USER}" make -s install-ckeditor-plugin
sudo -u "${INVENIO_WEB_USER}" make -s install-pdfa-helper-files
sudo -u "${INVENIO_WEB_USER}" make -s install-mediaelement
}
create_openoffice_tmp_space () {
sudo mkdir -p "${INVENIO_WEB_DSTDIR}/var/tmp/ooffice-tmp-files"
sudo chown -R nobody "${INVENIO_WEB_DSTDIR}/var/tmp/ooffice-tmp-files"
sudo chmod -R 755 "${INVENIO_WEB_DSTDIR}/var/tmp/ooffice-tmp-files"
}
configure_instance () {
# create invenio-local.conf
echo "[Invenio]
CFG_SITE_URL = http://${INVENIO_WEB_HOST}
CFG_SITE_SECURE_URL = https://${INVENIO_WEB_HOST}
CFG_DATABASE_HOST = ${INVENIO_MYSQL_HOST}
CFG_DATABASE_NAME = ${INVENIO_MYSQL_DBNAME}
CFG_DATABASE_USER = ${INVENIO_MYSQL_DBUSER}
CFG_DATABASE_PASS = ${INVENIO_MYSQL_DBPASS}
CFG_SITE_ADMIN_EMAIL = ${INVENIO_ADMIN_EMAIL}
CFG_SITE_SUPPORT_EMAIL = ${INVENIO_ADMIN_EMAIL}
CFG_WEBALERT_ALERT_ENGINE_EMAIL = ${INVENIO_ADMIN_EMAIL}
CFG_WEBCOMMENT_ALERT_ENGINE_EMAIL = ${INVENIO_ADMIN_EMAIL}
CFG_WEBCOMMENT_DEFAULT_MODERATOR = ${INVENIO_ADMIN_EMAIL}
CFG_BIBAUTHORID_AUTHOR_TICKET_ADMIN_EMAIL = ${INVENIO_ADMIN_EMAIL}
CFG_BIBCATALOG_SYSTEM_EMAIL_ADDRESS = ${INVENIO_ADMIN_EMAIL}
CFG_BIBSCHED_PROCESS_USER = ${INVENIO_WEB_USER}
CFG_MISCUTIL_SMTP_PORT = ${INVENIO_WEB_SMTP_PORT}
" | \
sudo -u "${INVENIO_WEB_USER}" tee "${INVENIO_WEB_DSTDIR}/etc/invenio-local.conf"
# update instance with this information:
sudo -u "${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --update-all
}
create_tables () {
sudo -u "${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --create-tables --yes-i-know
}
create_apache_configuration () {
sudo -u "${INVENIO_WEB_USER}" VIRTUAL_ENV="${VIRTUAL_ENV}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --create-apache-conf
}
-restart_apache_ubuntu_precise () {
- $sudo /etc/init.d/apache2 restart
+restart_apache_ubuntu12 () {
+ $sudo /usr/sbin/service apache2 restart
+}
+
+restart_apache_ubuntu14 () {
+ restart_apache_ubuntu12
}
restart_apache_centos6 () {
- $sudo /etc/init.d/httpd restart
+ $sudo /sbin/service httpd restart
}
main () {
# detect OS distribution and release version:
if hash lsb_release 2> /dev/null; then
os_distribution=$(lsb_release -i | cut -f 2)
os_release=$(lsb_release -r | cut -f 2 | grep -oE '[0-9]+\.' | cut -d. -f1 | head -1)
elif [ -e /etc/redhat-release ]; then
os_distribution=$(cut -d ' ' -f 1 /etc/redhat-release)
os_release=$(grep -oE '[0-9]+\.' /etc/redhat-release | cut -d. -f1 | head -1)
else
os_distribution="UNDETECTED"
os_release="UNDETECTED"
fi
# call appropriate provisioning functions:
if [ "$os_distribution" = "Ubuntu" ]; then
if [ "$os_release" = "12" ]; then
- create_apache_vhost_ubuntu_precise
+ create_apache_vhost_ubuntu12
+ create_symlinks
+ install_sources
+ create_openoffice_tmp_space
+ configure_instance
+ create_tables
+ create_apache_configuration
+ restart_apache_ubuntu12
+ elif [ "$os_release" = "14" ]; then
+ create_apache_vhost_ubuntu14
create_symlinks
install_sources
create_openoffice_tmp_space
configure_instance
create_tables
create_apache_configuration
- restart_apache_ubuntu_precise
+ restart_apache_ubuntu14
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
elif [ "$os_distribution" = "CentOS" ]; then
if [ "$os_release" = "6" ]; then
create_apache_vhost_centos6
create_symlinks
install_sources
create_openoffice_tmp_space
configure_instance
create_tables
create_apache_configuration
restart_apache_centos6
exit 1
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
else
echo "[ERROR] Sorry, unsupported distribution ${os_distribution}."
exit 1
fi
}
main
diff --git a/scripts/drop-instance.sh b/scripts/drop-instance.sh
index 8f0a81369..6b971a766 100755
--- a/scripts/drop-instance.sh
+++ b/scripts/drop-instance.sh
@@ -1,185 +1,204 @@
#!/usr/bin/env bash
#
# This file is part of Invenio.
# Copyright (C) 2015, 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# quit on errors:
set -o errexit
# check environment variables:
if [ "${INVENIO_MYSQL_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_MYSQL_HOST=192.168.50.11"
exit 1
fi
if [ "${INVENIO_MYSQL_DBNAME}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBNAME before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBNAME=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBUSER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBUSER before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBUSER=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBPASS}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBPASS before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBPASS=dbpass123"
exit 1
fi
if [ "${INVENIO_WEB_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_HOST=192.168.50.10"
exit 1
fi
if [ "${INVENIO_WEB_DSTDIR}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_DSTDIR before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_DSTDIR=/opt/invenio"
exit 1
fi
if [ "${INVENIO_WEB_USER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_USER before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_USER=www-data"
exit 1
fi
# quit on unbound symbols:
set -o nounset
# runs as root or needs sudo?
if [[ "$EUID" -ne 0 ]]; then
sudo='sudo'
else
sudo=''
fi
-start_apache_ubuntu_precise () {
- $sudo /etc/init.d/apache2 start
+start_apache_ubuntu12 () {
+ $sudo /usr/sbin/service apache2 start
}
-stop_apache_ubuntu_precise () {
- $sudo /etc/init.d/apache2 stop
+stop_apache_ubuntu12 () {
+ $sudo /usr/sbin/service apache2 stop
+}
+
+start_apache_ubuntu14 () {
+ start_apache_ubuntu12
+}
+
+stop_apache_ubuntu14 () {
+ stop_apache_ubuntu12
}
start_apache_centos6 () {
- $sudo /etc/init.d/httpd start
+ $sudo /sbin/service httpd start
}
stop_apache_centos6 () {
- $sudo /etc/init.d/httpd stop
+ $sudo /sbin/service httpd stop
}
-drop_apache_vhost_ubuntu_precise () {
- stop_apache_ubuntu_precise
+drop_apache_vhost_ubuntu12 () {
+ stop_apache_ubuntu12
if [ -e /etc/apache2/sites-available/default-ssl ]; then
$sudo /usr/sbin/a2ensite "*default*"
fi
if [ -L /etc/apache2/sites-enabled/invenio.conf ]; then
$sudo /usr/sbin/a2dissite "invenio*"
fi
- start_apache_ubuntu_precise
+ start_apache_ubuntu12
+}
+
+drop_apache_vhost_ubuntu14 () {
+ drop_apache_vhost_ubuntu12
}
drop_apache_vhost_centos6 () {
stop_apache_centos6
if grep -q "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf" /etc/httpd/conf/httpd.conf; then
sudo sed -i "s,^Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf,#Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost.conf,g" /etc/httpd/conf/httpd.conf
fi
if grep -q "Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf" /etc/httpd/conf/httpd.conf; then
sudo sed -i "s,^Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf,#Include ${INVENIO_WEB_DSTDIR}/etc/apache/invenio-apache-vhost-ssl.conf,g" /etc/httpd/conf/httpd.conf
fi
sudo sed -i 's,^#Alias /error/,Alias /error/,g' /etc/httpd/conf/httpd.conf
start_apache_centos6
}
drop_symlinks () {
for pythonversion in python2.4 python2.6 python2.7; do
for libversion in lib lib64 local/lib local/lib64; do
for packageversion in site-packages dist-packages; do
if [ -d /usr/$libversion/$pythonversion/$packageversion/ ] && [ ! -L /usr/$libversion/$pythonversion/$packageversion/invenio ]; then
$sudo rm /usr/$libversion/$pythonversion/$packageversion/invenio
fi
done
done
done
}
drop_instance_folder () {
$sudo rm -rf "${INVENIO_WEB_DSTDIR}/var/tmp/ooffice-tmp-files"
# shellcheck disable=SC2086
$sudo -u "${INVENIO_WEB_USER}" rm -rf ${INVENIO_WEB_DSTDIR}/*
}
drop_instance_tables () {
if [ -e "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" ]; then
$sudo -u "${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --drop-tables --yes-i-know
fi
}
main () {
# detect OS distribution and release version:
if hash lsb_release 2> /dev/null; then
os_distribution=$(lsb_release -i | cut -f 2)
os_release=$(lsb_release -r | cut -f 2 | grep -oE '[0-9]+\.' | cut -d. -f1 | head -1)
elif [ -e /etc/redhat-release ]; then
os_distribution=$(cut -d ' ' -f 1 /etc/redhat-release)
os_release=$(grep -oE '[0-9]+\.' /etc/redhat-release | cut -d. -f1 | head -1)
else
os_distribution="UNDETECTED"
os_release="UNDETECTED"
fi
# call appropriate provisioning functions:
if [ "$os_distribution" = "Ubuntu" ]; then
if [ "$os_release" = "12" ]; then
- stop_apache_ubuntu_precise
+ stop_apache_ubuntu12
+ drop_instance_tables
+ start_apache_ubuntu12
+ drop_apache_vhost_ubuntu12
+ drop_instance_folder
+ drop_symlinks
+ elif [ "$os_release" = "14" ]; then
+ stop_apache_ubuntu14
drop_instance_tables
- start_apache_ubuntu_precise
- drop_apache_vhost_ubuntu_precise
+ start_apache_ubuntu14
+ drop_apache_vhost_ubuntu14
drop_instance_folder
drop_symlinks
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
elif [ "$os_distribution" = "CentOS" ]; then
if [ "$os_release" = "6" ]; then
stop_apache_centos6
drop_instance_tables
start_apache_centos6
drop_apache_vhost_centos6
drop_instance_folder
drop_symlinks
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
else
echo "[ERROR] Sorry, unsupported distribution ${os_distribution}."
exit 1
fi
}
main
diff --git a/scripts/populate-instance.sh b/scripts/populate-instance.sh
index 0ad36b368..50c2ba4b6 100755
--- a/scripts/populate-instance.sh
+++ b/scripts/populate-instance.sh
@@ -1,125 +1,139 @@
#!/usr/bin/env bash
#
# This file is part of Invenio.
# Copyright (C) 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# quit on errors:
set -o errexit
# check environment variables:
if [ "${INVENIO_WEB_DSTDIR}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_DSTDIR before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_DSTDIR=/opt/invenio"
exit 1
fi
if [ "${INVENIO_WEB_USER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_USER before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_USER=www-data"
exit 1
fi
# quit on unbound symbols:
set -o nounset
# runs inside virtual environment?
VIRTUAL_ENV=${VIRTUAL_ENV:=}
# runs as root or needs sudo?
if [[ "$EUID" -ne 0 ]]; then
sudo='sudo'
else
sudo=''
fi
create_demo_site () {
$sudo -u "${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --create-demo-site --yes-i-know
}
load_demo_records () {
$sudo -u "${INVENIO_WEB_USER}" "${INVENIO_WEB_DSTDIR}/bin/inveniocfg" --load-demo-records --yes-i-know
}
apache_wsgi_restart () {
$sudo -u "${INVENIO_WEB_USER}" touch "${INVENIO_WEB_DSTDIR}/var/www-wsgi/invenio.wsgi"
}
-start_apache_ubuntu_precise () {
- $sudo /etc/init.d/apache2 start
+start_apache_ubuntu12 () {
+ $sudo /usr/sbin/service apache2 start
}
-stop_apache_ubuntu_precise () {
- $sudo /etc/init.d/apache2 stop
+stop_apache_ubuntu12 () {
+ $sudo /usr/sbin/service apache2 stop
+}
+
+start_apache_ubuntu14 () {
+ start_apache_ubuntu12
+}
+
+stop_apache_ubuntu14 () {
+ stop_apache_ubuntu12
}
start_apache_centos6 () {
- $sudo /etc/init.d/httpd start
+ $sudo /sbin/service httpd start
}
stop_apache_centos6 () {
- $sudo /etc/init.d/httpd stop
+ $sudo /sbin/service httpd stop
}
main () {
# detect OS distribution and release version:
if hash lsb_release 2> /dev/null; then
os_distribution=$(lsb_release -i | cut -f 2)
os_release=$(lsb_release -r | cut -f 2 | grep -oE '[0-9]+\.' | cut -d. -f1 | head -1)
elif [ -e /etc/redhat-release ]; then
os_distribution=$(cut -d ' ' -f 1 /etc/redhat-release)
os_release=$(grep -oE '[0-9]+\.' /etc/redhat-release | cut -d. -f1 | head -1)
else
os_distribution="UNDETECTED"
os_release="UNDETECTED"
fi
# call appropriate provisioning functions:
if [ "$os_distribution" = "Ubuntu" ]; then
if [ "$os_release" = "12" ]; then
- stop_apache_ubuntu_precise
+ stop_apache_ubuntu12
+ create_demo_site
+ start_apache_ubuntu12
+ load_demo_records
+ apache_wsgi_restart
+ elif [ "$os_release" = "14" ]; then
+ stop_apache_ubuntu14
create_demo_site
- start_apache_ubuntu_precise
+ start_apache_ubuntu14
load_demo_records
apache_wsgi_restart
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
elif [ "$os_distribution" = "CentOS" ]; then
if [ "$os_release" = "6" ]; then
stop_apache_centos6
create_demo_site
start_apache_centos6
load_demo_records
apache_wsgi_restart
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
else
echo "[ERROR] Sorry, unsupported distribution ${os_distribution}."
exit 1
fi
}
main
diff --git a/scripts/provision-mysql.sh b/scripts/provision-mysql.sh
index d21d8ea19..51454ae12 100755
--- a/scripts/provision-mysql.sh
+++ b/scripts/provision-mysql.sh
@@ -1,163 +1,169 @@
#!/usr/bin/env bash
#
# This file is part of Invenio.
# Copyright (C) 2015, 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# quit on errors:
set -o errexit
# check environment variables:
if [ "${INVENIO_MYSQL_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_MYSQL_HOST=192.168.50.11"
exit 1
fi
if [ "${INVENIO_MYSQL_DBNAME}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBNAME before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBNAME=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBUSER}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBUSER before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBUSER=invenio1"
exit 1
fi
if [ "${INVENIO_MYSQL_DBPASS}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_MYSQL_DBPASS before runnning this script."
echo "[ERROR] Example: INVENIO_MYSQL_DBPASS=dbpass123"
exit 1
fi
if [ "${INVENIO_WEB_HOST}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_WEB_HOST before runnning this script."
echo "[ERROR] Example: export INVENIO_WEB_HOST=192.168.50.10"
exit 1
fi
# quit on unbound symbols:
set -o nounset
-provision_mysql_ubuntu_precise () {
+provision_mysql_ubuntu12 () {
# update list of available packages
sudo DEBIAN_FRONTEND=noninteractive apt-get update
# install MySQL server:
sudo DEBIAN_FRONTEND=noninteractive apt-get -y install \
mysql-server
# allow network connections:
if ! grep -q "${INVENIO_MYSQL_HOST}" /etc/mysql/my.cnf; then
sudo sed -i "s/127.0.0.1/${INVENIO_MYSQL_HOST}/" /etc/mysql/my.cnf
fi
# restart MySQL server:
sudo /usr/sbin/service mysql restart
}
+provision_mysql_ubuntu14 () {
+ provision_mysql_ubuntu12
+}
+
provision_mysql_centos6 () {
# update list of available packages
sudo yum update -y
# install MySQL server:
sudo yum install -y \
mysql-server
# open firewall ports:
if [ -e /sbin/iptables ]; then
thisinputchain="INPUT"
if sudo /sbin/iptables -nL | grep -q 'Chain RH-Firewall-1-INPUT'; then
thisinputchain="RH-Firewall-1-INPUT"
fi
if ! sudo /sbin/iptables -nL | grep -q dpt:3306; then
sudo /sbin/iptables -I $thisinputchain -p tcp -m tcp --dport 3306 -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp -m tcp --dport 3306 -j ACCEPT
fi
if ! sudo /sbin/iptables -nL | grep -q dpt:3306; then
sudo /sbin/iptables -I $thisinputchain -p tcp -m tcp --dport 3306 -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp -m tcp --dport 3306 -j ACCEPT
fi
fi
# save new firewall rules to survive reboot:
- sudo /etc/init.d/iptables save
+ sudo /sbin/service iptables save
# enable MySQL upon reboot:
sudo /sbin/chkconfig mysqld on
# restart MySQL server:
- sudo /etc/init.d/mysqld restart
+ sudo /sbin/service mysqld restart
}
setup_db () {
# create database if it does not exist:
echo "CREATE DATABASE IF NOT EXISTS ${INVENIO_MYSQL_DBNAME} DEFAULT CHARACTER SET utf8;" | \
mysql -u root -B
# grant privileges to the user on this database:
echo "GRANT ALL PRIVILEGES ON ${INVENIO_MYSQL_DBNAME}.* TO ${INVENIO_MYSQL_DBUSER}@${INVENIO_WEB_HOST} IDENTIFIED BY '${INVENIO_MYSQL_DBPASS}';" | \
mysql -u root -B
}
main () {
# detect OS distribution and release version:
if hash lsb_release 2> /dev/null; then
os_distribution=$(lsb_release -i | cut -f 2)
os_release=$(lsb_release -r | cut -f 2 | grep -oE '[0-9]+\.' | cut -d. -f1 | head -1)
elif [ -e /etc/redhat-release ]; then
os_distribution=$(cut -d ' ' -f 1 /etc/redhat-release)
os_release=$(grep -oE '[0-9]+\.' /etc/redhat-release | cut -d. -f1 | head -1)
else
os_distribution="UNDETECTED"
os_release="UNDETECTED"
fi
# call appropriate provisioning functions:
if [ "$os_distribution" = "Ubuntu" ]; then
if [ "$os_release" = "12" ]; then
- provision_mysql_ubuntu_precise
+ provision_mysql_ubuntu12
+ elif [ "$os_release" = "14" ]; then
+ provision_mysql_ubuntu14
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
elif [ "$os_distribution" = "CentOS" ]; then
if [ "$os_release" = "6" ]; then
provision_mysql_centos6
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
else
echo "[ERROR] Sorry, unsupported distribution ${os_distribution}."
exit 1
fi
# finish with common setups:
setup_db
}
main
diff --git a/scripts/provision-web.sh b/scripts/provision-web.sh
index 8422bd89b..c48a67541 100755
--- a/scripts/provision-web.sh
+++ b/scripts/provision-web.sh
@@ -1,262 +1,347 @@
#!/usr/bin/env bash
#
# This file is part of Invenio.
# Copyright (C) 2015, 2016 CERN.
#
# Invenio is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307, USA.
#
# In applying this license, CERN does not
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.
# quit on errors:
set -o errexit
# check environment variables:
if [ "${INVENIO_SRCDIR}" = "" ]; then
echo "[ERROR] Please set environment variable INVENIO_SRCDIR before runnning this script."
echo "[ERROR] Example: export INVENIO_SRCDIR=/vagrant"
exit 1
fi
# quit on unbound symbols:
set -o nounset
# runs inside virtual environment?
VIRTUAL_ENV=${VIRTUAL_ENV:=}
# runs as root or needs sudo?
if [[ "$EUID" -ne 0 ]]; then
sudo='sudo'
else
sudo=''
fi
# unattended installation:
export DEBIAN_FRONTEND=noninteractive
-provision_web_ubuntu_precise () {
+provision_web_ubuntu12 () {
# update list of available packages
$sudo DEBIAN_FRONTEND=noninteractive apt-get update
# install useful system packages
$sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
apache2-mpm-worker \
automake \
clisp \
curl \
cython \
gettext \
giflib-tools \
git \
- gnuplot poppler-utils \
+ gnuplot \
html2text \
ipython \
libapache2-mod-wsgi \
libapache2-mod-xsendfile \
libffi-dev \
libfreetype6-dev \
libjpeg-dev \
libmsgpack-dev \
libmysqlclient-dev \
libpng-dev \
libssl-dev \
libtiff-dev \
libxml2-dev \
libxslt-dev \
make \
+ mlocate \
mysql-client \
netpbm \
openOffice.org \
pdftk \
pep8 \
+ poppler-utils \
postfix \
pstotext \
pylint \
python-dev \
python-gnuplot \
python-h5py \
python-libxml2 \
python-libxslt1 \
python-nose \
python-nosexcover \
python-pip \
python-uno \
redis-server \
rlwrap \
sbcl \
screen \
texlive \
unzip \
vim
# make sure Apache is started:
sudo /usr/sbin/service apache2 restart
# make sure Redis is started:
sudo /usr/sbin/service redis-server restart
+ # update list of files e.g. useful for Apache version detection:
+ sudo updatedb
+
+ # grant Apache user the nobody user rights for OpenOffice integration:
+ echo "www-data ALL=(nobody) NOPASSWD: ALL" | $sudo tee /etc/sudoers.d/www-data
+ $sudo chmod o-r /etc/sudoers.d/www-data
+
+}
+
+provision_web_ubuntu14 () {
+
+ # update list of available packages
+ $sudo DEBIAN_FRONTEND=noninteractive apt-get update
+
+ # install useful system packages
+ $sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
+ apache2-mpm-worker \
+ automake \
+ clisp \
+ curl \
+ cython \
+ gettext \
+ giflib-tools \
+ git \
+ gnuplot \
+ html2text \
+ ipython \
+ libapache2-mod-wsgi \
+ libapache2-mod-xsendfile \
+ libffi-dev \
+ libfreetype6-dev \
+ libjpeg-dev \
+ libmsgpack-dev \
+ libmysqlclient-dev \
+ libpng-dev \
+ libssl-dev \
+ libtiff-dev \
+ libxml2-dev \
+ libxslt-dev \
+ make \
+ mlocate \
+ mysql-client \
+ netpbm \
+ libreoffice \
+ pdftk \
+ pep8 \
+ poppler-utils \
+ postfix \
+ pstotext \
+ pylint \
+ python-dev \
+ python-gnuplot \
+ python-h5py \
+ python-libxml2 \
+ python-libxslt1 \
+ python-nose \
+ python-nosexcover \
+ python-pip \
+ libreoffice-script-provider-python \
+ redis-server \
+ rlwrap \
+ sbcl \
+ screen \
+ texlive \
+ unzip \
+ vim
+
+ # make sure Apache is started:
+ sudo /usr/sbin/service apache2 restart
+
+ # make sure Redis is started:
+ sudo /usr/sbin/service redis-server restart
+
+ # update list of files e.g. useful for Apache version detection:
+ sudo updatedb
+
# grant Apache user the nobody user rights for OpenOffice integration:
echo "www-data ALL=(nobody) NOPASSWD: ALL" | $sudo tee /etc/sudoers.d/www-data
$sudo chmod o-r /etc/sudoers.d/www-data
}
provision_web_centos6 () {
# update list of available packages
$sudo yum update -y
# add EPEL external repository:
$sudo yum install -y epel-release
# install useful system tools:
$sudo yum install -y \
automake \
curl \
cython \
file \
freetype-devel \
gcc \
gcc-c++ \
gettext \
gettext-devel \
git \
gnuplot-py \
h5py \
hdf5-devel \
ipython \
libffi-devel \
libpng-devel \
libreoffice \
libreoffice-headless \
libreoffice-pyuno \
libxml2-devel \
libxml2-python \
libxslt-devel \
libxslt-python \
+ mlocate \
mod_ssl \
mod_wsgi \
mysql-devel \
poppler-utils \
python-devel \
python-pip \
redis \
rlwrap \
screen \
sendmail \
sudo \
texlive \
unzip \
vim \
w3m \
wget
# open firewall ports:
if [ -e /sbin/iptables ]; then
thisinputchain="INPUT"
if sudo /sbin/iptables -nL | grep -q 'Chain RH-Firewall-1-INPUT'; then
thisinputchain="RH-Firewall-1-INPUT"
fi
if ! sudo /sbin/iptables -nL | grep -q dpt:http; then
sudo /sbin/iptables -I $thisinputchain -p tcp -m tcp --dport 80 -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp -m tcp --dport 80 -j ACCEPT
fi
if ! sudo /sbin/iptables -nL | grep -q dpt:https; then
sudo /sbin/iptables -I $thisinputchain -p tcp -m tcp --dport 443 -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp -m tcp --dport 443 -j ACCEPT
fi
fi
# save new firewall rules to survive reboot:
- sudo /etc/init.d/iptables save
+ sudo /sbin/service iptables save
# enable Apache upon reboot:
sudo /sbin/chkconfig httpd on
# enable Redis upon reboot:
sudo /sbin/chkconfig redis on
# make sure Apache is started:
- sudo /etc/init.d/httpd restart
+ sudo /sbin/service httpd restart
# make sure Redis is started:
- sudo /etc/init.d/redis restart
+ sudo /sbin/service redis restart
+
+ # update list of files e.g. useful for Apache version detection:
+ sudo updatedb
# grant Apache user the nobody user rights for OpenOffice integration:
echo "apache ALL=(nobody) NOPASSWD: ALL" | $sudo tee /etc/sudoers.d/apache
$sudo chmod o-r /etc/sudoers.d/apache
}
provision_web_pypi () {
# install Python packages from PyPI
olddir=$(pwd)
cd "${INVENIO_SRCDIR}"
for reqfile in requirements*.txt; do
if [ -e "$reqfile" ]; then
if [ "$VIRTUAL_ENV" != "" ]; then
pip install -r "$reqfile"
else
sudo pip install -r "$reqfile"
fi
fi
done
cd "${olddir}"
}
main () {
# detect OS distribution and release version:
if hash lsb_release 2> /dev/null; then
os_distribution=$(lsb_release -i | cut -f 2)
os_release=$(lsb_release -r | cut -f 2 | grep -oE '[0-9]+\.' | cut -d. -f1 | head -1)
elif [ -e /etc/redhat-release ]; then
os_distribution=$(cut -d ' ' -f 1 /etc/redhat-release)
os_release=$(grep -oE '[0-9]+\.' /etc/redhat-release | cut -d. -f1 | head -1)
else
os_distribution="UNDETECTED"
os_release="UNDETECTED"
fi
# call appropriate provisioning functions:
if [ "$os_distribution" = "Ubuntu" ]; then
if [ "$os_release" = "12" ]; then
- provision_web_ubuntu_precise
+ provision_web_ubuntu12
+ provision_web_pypi
+ elif [ "$os_release" = "14" ]; then
+ provision_web_ubuntu14
provision_web_pypi
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
elif [ "$os_distribution" = "CentOS" ]; then
if [ "$os_release" = "6" ]; then
provision_web_centos6
provision_web_pypi
else
echo "[ERROR] Sorry, unsupported release ${os_release}."
exit 1
fi
else
echo "[ERROR] Sorry, unsupported distribution ${os_distribution}."
exit 1
fi
}
main