-------------------------------------------------------------------- CDS Invenio v0.99.0 is released March 27, 2008 http://cdsware.cern.ch/invenio/news.html -------------------------------------------------------------------- CDS Invenio v0.99.0 was released on March 27, 2008. What's new: ----------- *) new Invenio configuration language, new inveniocfg configuration tool permitting more runtime changes and enabling separate local customizations (MiscUtil) *) phased out WML dependency everywhere (all modules) *) new common RSS cache implementation (WebSearch) *) improved access control to the detailed record pages (WebSearch) *) when searching non-existing collections, do not revert to searching in public Home anymore (WebSearch) *) strict calculation of number of hits per multiple collections (WebSearch) *) propagate properly language environment in browse pages, thanks to Ferran Jorba (WebSearch) *) search results sorting made accentless, thanks to Ferran Jorba (WebSearch) *) new OpenURL interface (WebSearch) *) added new search engine API argument to limit searches to record creation/modification dates and times instead of hitherto creation dates only (WebSearch) *) do not allow HTTP POST method for searches to prevent hidden mining (WebSearch) *) added alert and RSS teaser for search engine queries (WebSearch) *) new optimized index structure for fast integer bit vector operations, leading to significant indexing time improvements (MiscUtil, BibIndex, WebSearch) *) new tab-based organisation of detailed record pages, with new URL schema (/record/1/usage) and related CSS changes (BibFormat, MiscUtil, WebComment, WebSearch, WebStyle, WebSubmit) *) phased out old PHP based code; migration to Python-based output formats recommended (BibFormat, WebSubmit) *) new configurability to show/hide specific output formats for specific collections (BibFormat, WebSearch) *) new configurability to have specific stemming settings for specific indexes (BibIndex, WebSearch) *) optional removal of LaTeX markup for indexer (BibIndex, WebSearch) *) performance optimization for webcoll and optional arguments to refresh only parts of collection cache (WebSearch) *) optional verbosity argument propagation to the output formatter (BibFormat, WebSearch) *) new convenient reindex option to the indexer (BibIndex) *) fixed problem with indexing of some lengthy UTF-8 accented names, thanks to Theodoros Theodoropoulos for reporting the problem (BibIndex) *) fixed full-text indexing of HTML pages (BibIndex) *) new Stemmer module dependency, fixes issues on 64-bit systems (BibIndex) *) fixed download history graph display (BibRank) *) improved citation ranking and history graphs, introduced self-citation distinction, added new demo records (BibRank) *) fixed range redefinition and output message printing problems in the ranking indexer, thanks to Mike Marino (BibRank) *) new XSLT output formatter support; phased out old BFX formats (BibFormat) *) I18N output messages are now translated in the output formatter templates (BibFormat) *) formats fixed to allow multiple author affiliations (BibFormat) *) improved speed of the record output reformatter in case of large sets (BibFormat) *) support for displaying LaTeX forumas via JavaScript (BibFormat) *) new and improved output formatter elements (BibFormat) *) new escaping modes for format elements (BibFormat) *) output format template editor cache and element dependency checker improvements (BibFormat) *) output formatter speed improvements in PHP-compatible mode (BibFormat) *) new demo submission configuration and approval workflow examples (WebSubmit) *) new submission full-text file stamper utility (WebSubmit) *) new submission icon-creation utility (WebSubmit) *) separated submission engine and database layer (WebSubmit) *) submission functions can now access user information (WebSubmit) *) implemented support for restricted icons (WebSubmit, WebAccess) *) new full-text file URL and cleaner storage facility; requires file names to be unique within a given record (WebSearch, WebSubmit) *) experimental release of the complex approval and refereeing workflow (WebSubmit) *) new end-submission functions to move files to storage space (WebSubmit) *) added support for MD5 checking of full-text files (WebSubmit) *) improved behaviour of the submission system with respect to the browser "back" button (WebSubmit) *) removed support for submission "cookies" (WebSubmit) *) flexible report number generation during submission (WebSubmit) *) added support for optional filtering step in the OAI harvesting chain (BibHarvest) *) new text-oriented converter functions IFDEFP, JOINMULTILINES (BibConvert) *) selective harvesting improvements, sets, non-standard responses, safer resumption token handling (BibHarvest) *) OAI archive configuration improvements: collections retrieval, multiple set definitions, new clean mode, timezones, and more (BibHarvest) *) OAI gateway improvements: XSLT used to produce configurable output (BibHarvest) *) added support for "strong tags" that can resist metadata replace mode (BibUpload) *) added external OAI ID tag support to the uploader (BibUpload) *) added support for full-text file transfer during uploading (BibUpload) *) preserving full history of all MARCXML versions of a record (BibEdit, BibUpload) *) XMLMARC to TextMarc improvements: empty indicators and more (BibEdit) *) numerous reference extraction tool improvements: year handling, LaTeX handling, URLs, journal titles, output methods, and more (BibEdit) *) new classification daemon (BibClassify) *) classification taxonomy caching resulting in speed optimization (BibClassify) *) new possibility to define more than one keyword taxonomy per collection (BibClassify) *) fixed non-standalone keyword detection, thanks to Annette Holtkamp (BibClassify) *) new embedded page generation profiler (WebStyle) *) new /help pages layout and webdoc formatting tool (WebStyle) *) new custom style template verification tool (WebStyle) *) added support for the XML page() output format, suitable for AJAX interfaces (WebStyle) *) introduction of navigation menus (WebStyle) *) general move from HTML to XHTML markup (all modules) *) fixed alert deletion tool vulnerability (WebAlert) *) do not advertise baskets/alerts much for guest users; show only the login link (WebSession) *) password reset interface improvements (WebSession) *) new permanent "remember login" mechanism (WebSession, WebAccess) *) local user passwords are now encrypted (WebSession, WebAccess) *) new LDAP external authentication plugin (WebAccess) *) new password reset mechanism using new secure mail cookies and temporary role membership facilities (WebAccess, WebSession) *) added support for Single Sign-On Shibboleth based authentication method (WebAccess) *) new firewall-like based role definition language, new demo examples (WebAccess) *) external authentication and groups improvements: nicknames, account switching, and more (WebSession, WebAccess) *) task log viewer integrated in the task monitor (BibSched) *) new journal creation module (WebJournal) *) new generic statistic gathering and display facility (WebStat) *) deployed new common email sending facility (MiscUtil, WebAlert, WebComment, WebSession, WebSubmit) *) dropped support for MySQL-4.0, permitting to use clean and strict UTF-8 storage methods; upgrade of MySQLdb to at least 1.2.1_p2 required (MiscUtil) *) uncatched exceptions are now being sent by email to the administrator (MiscUtil, WebStyle) *) new general garbage collector with a possibility to run via the task scheduler and a possibility to clean unreferenced bibliographic values (MiscUtil) *) new generic SQL and data cacher (MiscUtil) *) new HTML page validator plugin (MiscUtil) *) new web test suite running in a real browser (MiscUtil) *) improved code kwalitee checker (MiscUtil) *) translation updates: Spanish and Catalan (thanks to Ferran Jorba), Japanese (Toru Tsuboyama), German (Benedikt Koeppel), Polish (Zbigniew Szklarz and Zbigniew Leonowicz), Greek (Theodoros Theodoropoulos), Russian (Yana Osborne), Swedish, Italian, French *) new translations: Chinese traditional and Chinese simplified (thanks to Kam-ming Ku) *) ... plus many other minor bug fixes and improvements Download: --------- Installation notes: ------------------- Please follow the INSTALL file bundled in the distribution tarball. Upgrade notes: -------------- If you are upgrading from CDS Invenio v0.92.1, then we advise you to try the upgrade first on a development server. There was a number of important changes since the last stable release that might make the upgrade possibly time-consuming in case you have to dump and reload your bibliographic tables. Please follow the following steps: - Launch the bibsched monitor and wait until all active bibsched tasks are finished. Then put bibsched daemon into manual mode. - Stop all submission procedures and other write operations. For example, you may want to stop Apache, edit httpd.conf to introduce a global site redirect to a temporary splash page saying that upgrade is in progress, and restart Apache. - Take a backup of your current MySQL database and your CDS Invenio installation directory (usually /opt/cds-invenio). Remove your old /opt/cds-invenio/lib/python/invenio/magic and /opt/cds-invenio/var/www/help directories. - First of all, note that CDS Invenio v0.99.0 must use MySQL server at least 4.1 and the database must be running in UTF-8 mode. If you have been running older MySQL server, or you have not created your database and data in default charset UTF-8 but say in Latin-1, then you must dump and reload all your tables. In order to check which version and charset you have been using, you can run: $ echo "SELECT VERSION()" | /opt/cds-invenio/bin/dbexec $ echo "SHOW CREATE DATABASE cdsinvenio" | /opt/cds-invenio/bin/dbexec $ echo "SHOW CREATE TABLE bib10x" | /opt/cds-invenio/bin/dbexec If you need to convert your tables to UTF-8, then you basically have to dump the tables, change encoding of the file, and load the content back into new table definitions. This process may take some time and carefulness. See for example this blog post: . - Second of all, you must upgrade some Python modules, namely your MySQLdb module as indicated in the INSTALL file. Note that this will make your current site unusable, but we have stopped Apache already anyway. If you have been using Stemmer, you must upgrade its version as indicated in the INSTALL file. - Untar new sources and rerun configure with old prefix argument (your old configure line is available at /opt/cds-invenio-BACKUP/etc/build/config.nice) and install CDS Invenio (see INSTALL file, part 1a). - CDS Invenio v0.99.0 uses a new INI-style configuration file invenio.conf instead of the old config.wml. Please see INSTALL file, part 1b on how to customize your installation. You will have to create a 'invenio-local.conf' file where you have to merge your old config.wml values (they are available at /opt/cds-invenio-BACKUP/lib/wml/invenio/config.wml). Beware of variable name updates, e.g. 'filedirsize' now became 'CFG_WEBSUBMIT_FILESYSTEM_BIBDOC_GROUP_LIMIT'. - If you have previously customized your page header and footer via WML, then you should now put your page customizations in a new template skin. Please see the WebStyle Admin Guide for more information. Also, if you have edited v0.92.1 CSS style sheet, you may want to merge your changes into new elements of the 0.99.0 style sheet. - If you have customized your /opt/cds-invenio/etc/ files in the last installation (for example to edit the stopwords list or configure language stemmers), then you have to restore your changes into corresponding /opt/cds-invenio/etc files. - Update your database table structure: $ make update-v0.92.1-tables If you are upgrading from previous CDSware releases such as v0.7.1, then you may need to run more update statements of this kind, as indicated in previous RELEASE-NOTES files. (You could also start afresh, copying your old data into a fresh v0.99.0 installation.) - If you have previously customized your page templates (e.g. webbasket_templates.py), then check for changes the new version brings: $ make check-custom-templates This script will check if your customized page templates (see the WebStyle Admin Guide) still conform to the default templates. This gives you a chance to fix your templates before running 'make install', by providing a list of incompatibilities of your custom templates with the new versions of the templates. This step is only useful if you are upgrading and if you have previously customized default page templates. - You have also to run manually several migration scripts: $ python ./modules/webaccess/lib/collection_restrictions_migration_kit.py This script will migrate restricted collections that used Apache user groups to a new firewall-like role definition language. $ python ./modules/websubmit/lib/fulltext_files_migration_kit.py This script will check and update your fulltext file storage system to the new style (e.g. unique document names). $ python ./modules/websession/lib/password_migration_kit.py This script will update your local user table in order to use encrypted passwords for more security. - CDS Invenio v0.99.0 uses new, faster word indexer. You will have to reindex all your indexes by running: $ /opt/cds-invenio/bin/bibindex -u admin -R - You have to edit your httpd.conf in order to make Apache aware of new URLs. Please run: $ /opt/cds-invenio/bin/inveniocfg --create-apache-conf and check differences with your current setup and edit as appropriate. - Restart Apache and check whether everything is alright with your system. Note that the detailed record pages (/record/10) have now a tab-style look by default. You may want to update your formats to this style or else to disable this feature. Please see the WebStyle Admin Guide for more information. - Put the bibsched daemon back into the automatic mode. You are done. Further notes and issues: ------------------------- *) Some modules of this release (e.g. mail submission system) are still experimental and not yet activated. You may have a peek at what is planned, but please do not rely on them. *) The admin-level functionality of several modules is not fully developed or documented yet. What's next: ------------ *) Improving the known issues mentioned above. Strengthening the documentation towards the v1.0.0 release. *) Improving the inveniocfg tool with automated upgrade capabilities. *) Deploying the new URL schema for all pages (admin). - end of file -