Diffusion invenio-infoscience ab532de38f71

BibIndex: pluginutils for tokenizers
ab532de38f71
Actions

Authored by Grzegorz Szpura <grzegorz.szpura@cern.ch> on Apr 9 2013, 14:13.

Description

BibIndex: pluginutils for tokenizers

CFG_BIBINDEX_PATH_TO_STOPWORDS_FILE now is a CFG_BIBRANK_PATH_TO_STOPWORDS_FILE.
Indexes have their own path to stopwords file.
Tokenizers are loaded with pluginutils. (references #852)

Signed-off-by: Grzegorz Szpura <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>

Details

Committed

Tibor Simko <tibor.simko@cern.ch>

Oct 3 2013, 23:08

Parents

R3600:b0a11546ca41: BibIndex: centralisation of tokenizers

Branches

Unknown

Tags

Unknown

Event Timeline

Tibor Simko <tibor.simko@cern.ch> committed R3600:ab532de38f71: BibIndex: pluginutils for tokenizers (authored by Grzegorz Szpura <grzegorz.szpura@cern.ch>).Oct 3 2013, 23:08

Changes (40)

		Path
M		config/invenio-autotools.conf.in
M		configure.ac
M		modules/bibindex/lib/Makefile.am
M		modules/bibindex/lib/bibindex_engine.py
M		modules/bibindex/lib/bibindex_engine_config.py
M		modules/bibindex/lib/bibindex_engine_stopwords.py
D		modules/bibindex/lib/bibindex_engine_tokenizer.py
M		modules/bibindex/lib/bibindex_engine_tokenizer_unit_tests.py
M		modules/bibindex/lib/bibindex_engine_unit_tests.py
M		modules/bibindex/lib/bibindex_engine_utils.py
M		modules/bibindex/lib/bibindex_engine_washer.py
M		modules/bibindex/lib/bibindex_regression_tests.py
M		modules/bibindex/lib/bibindexadmin_regression_tests.py
M		modules/bibindex/lib/bibindexadminlib.py
A	(dir)	modules/bibindex/lib/tokenizers/
A		modules/bibindex/lib/tokenizers/BibIndexAuthorCountTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexAuthorTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexDefaultTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexEmptyTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexExactAuthorTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexFulltextTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexJournalTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexTokenizer.py
A		modules/bibindex/lib/tokenizers/BibIndexYearTokenizer.py
P		modules/bibindex/lib/tokenizers/Makefile.am Copied from modules/bibsort/lib/Makefile.am
P		modules/bibindex/lib/tokenizers/__init__.py Copied from modules/bibauthorid/lib/__init__.py
M		modules/bibrank/lib/bibrank_citation_indexer.py
M		modules/bibrank/lib/bibrank_record_sorter.py
M		modules/bibrank/lib/bibrank_word_indexer.py
M		modules/bibrank/lib/bibrank_word_searcher.py
M		modules/bibupload/lib/bibupload_regression_tests.py
M		modules/docextract/lib/refextract_api.py
M		modules/docextract/lib/refextract_linker.py
P		modules/miscutil/lib/upgrades/invenio_2013_03_29_idxINDEX_stopwords_update.py Copied from modules/miscutil/lib/upgrades/invenio_2013_03_21_idxINDEX_stopwords.py
M		modules/miscutil/sql/tabcreate.sql
M		modules/websearch/lib/search_engine.py
M		modules/websearch/lib/search_engine_query_parser.py
M		modules/webstat/lib/webstat.py
M		modules/webstat/lib/webstat_engine.py
M		modules/webstat/lib/webstat_webinterface.py