Diffusion invenio-infoscience (maint-1.0) History

R3600/modules/bibindex/lib/bibindex_engine.py
maint-1.0

Jun 1 2016

global: silencing MySQL UTF-8 warnings for BLOBs
cdbe6b465302
·Tibor Simko on Jun 1 2016, 11:06.

Sep 28 2012

global: removal of psyco
1795fd0ffa53
·Samuele Kaplun / Tibor Simko on Sep 28 2012, 11:27.

Feb 24 2012

BibIndex: fix author:Campbell-Wilson word query
657c4701b98a
·Tibor Simko on Feb 24 2012, 09:41.

Feb 13 2012

Merge branch 'maint-0.99' into maint-1.0
db31e01cefa6
·Tibor Simko on Feb 13 2012, 13:11.

Sep 12 2011

BibIndex: fix double-stemming upon indexing
4f37b87ffdd3
·Samuele Kaplun / Tibor Simko on Sep 12 2011, 21:34.

Dec 13 2010

installation: new firstauthor index
a3ac0436fe79
·Tibor Simko on Dec 13 2010, 10:27.

Dec 9 2010

BibIndex: wash input phrases for UTF-8
3ee31c4d53bc
·Tibor Simko on Dec 9 2010, 10:17.

Dec 8 2010

BibIndex: special treatment for arXiv:NNNN.NNNN
ee2109cfe7ac
·Tibor Simko on Dec 8 2010, 22:31.
BibIndex: less verbose fulltext indexing
59ae79a35de9
·Samuele Kaplun / Tibor Simko on Dec 8 2010, 11:20.

Nov 19 2010

BibIndex: author index include/exclude first names
94b430fc3201
·Tibor Simko on Nov 19 2010, 13:40.

Nov 1 2010

global: update copyright years
8f0027eea7ee
·Tibor Simko on Nov 1 2010, 23:50.
global: CDS Invenio becomes Invenio
d30e56fa001a
·Tibor Simko on Nov 1 2010, 22:08.
BibIndex: fix wrong idxPAIR term washing
72ed66546076
·Samuele Kaplun / Tibor Simko on Nov 1 2010, 10:39.

Aug 17 2010

kwalitee: adapt disable-msg to new pylint
3c3dc2e7fcc5
·Tibor Simko on Aug 17 2010, 22:36.
Removal of unused imports
d277958c9e4d
·Benoit Thiell / Tibor Simko on Aug 17 2010, 19:25.

May 26 2010

BibIndex: raise meaningful exceptions
e2feb8f6e2ee
·Benoit Thiell / Tibor Simko on May 26 2010, 11:48.

May 25 2010

BibIndex: fix remote full-text file indexing
372bfd0ce949
·Samuele Kaplun / Tibor Simko on May 25 2010, 16:52.

May 17 2010

BibIndex: index files with unusual extensions
6c9e98b941e2
·Samuele Kaplun / Tibor Simko on May 17 2010, 10:25.

Apr 14 2010

BibIndex: quick hack to index hidden files
a5a8668bf77d
·Samuele Kaplun / Tibor Simko on Apr 14 2010, 19:38.

Apr 13 2010

BibIndex: added support for idxPAIR
d54e912fa70e
·Samuele Kaplun on Apr 13 2010, 09:26.
WebSubmit: new fulltext conversion tools
7b3804d29a49
·Samuele Kaplun on Apr 13 2010, 09:26.

Mar 28 2010

BibIndex: created exact and fuzzy author indexes
bdd48973620d
·Tibor Simko on Mar 28 2010, 19:04.

Mar 8 2010

BibIndex: new get_words_from_date_tag()
f4c91b2e7a13
·Tibor Simko on Mar 8 2010, 16:04.

Nov 30 2009

BibIndex: do not email exceptions when repairing
3a120cbd10ae
·Samuele Kaplun / Tibor Simko on Nov 30 2009, 22:07.

Aug 25 2009

BibIndex: more explicit messages during reindexing
98a733305a94
·Tibor Simko on Aug 25 2009, 21:35.

Jun 26 2009

htmlutils: new remove_html_markup() API function
1c3f5175204b
·Tibor Simko on Jun 26 2009, 17:23.

Jun 15 2009

Global move from javadoc to epydoc docstring style
40394efc1646
·Tibor Simko on Jun 15 2009, 18:02.

Jan 8 2009

Amended and merged branch sam/idxPHRASE-deployment
dfa81009a5b7
·Tibor Simko on Jan 8 2009, 17:31.

Dec 8 2008

Improved index corruption emergency help messages.
042fd85e1ace
·Tibor Simko on Dec 8 2008, 18:12.

Oct 31 2008

Disable phrase breaking algorithm.
e28cd2bc7cd5
·Samuele Kaplun on Oct 31 2008, 16:18.
Correctly checking for proper use of --reindex.
cc46dc12bf20
·Samuele Kaplun on Oct 31 2008, 16:18.
Fixed wrong call to wordTable.update_last_updated.
ac97c25cc183
·Samuele Kaplun on Oct 31 2008, 16:18.
Implemented reindexing via temporary table swapping.
2025e6af3243
·Samuele Kaplun on Oct 31 2008, 16:18.
Disabled stemming for idxPHRASE.
a40f144704d3
·Samuele Kaplun on Oct 31 2008, 16:16.
Plugged back idxPHRASE creation.
d60b499c3006
·Samuele Kaplun on Oct 31 2008, 16:16.

Oct 27 2008

Removed $Id$ from file preamble comments.
51bfbcb9fa42
·Tibor Simko on Oct 27 2008, 22:39.

Sep 9 2008

Fixing bug (reported by Devin Bougie) triggered when filename has no extension…
919c5c448915
·Samuele Kaplun on Sep 9 2008, 19:29.

Aug 12 2008

Escaped filenames. in get_words_from_fulltext
ac6d47d5fa26
·Samuele Kaplun on Aug 12 2008, 12:03.

Jun 17 2008

Catch exceptions of lower_index_term() and continue in spite of UTF-8 problems…
6c4510b06391
·Tibor Simko on Jun 17 2008, 16:34.

Jun 4 2008

Cosmetics of output messages for the stemming info.
c3b76be701ba
·Tibor Simko on Jun 4 2008, 19:46.

May 29 2008

Lower phrase-to-be-indexed in a Unicode-friendly manner.
62b87aa91c88
·Tibor Simko on May 29 2008, 23:54.

May 16 2008

First implementation of the intelligent 'journal' index. This is an index…
4dd00adc2995
·Tibor Simko on May 16 2008, 14:53.

May 14 2008

Better closed mkstemp-created files.
e12a89b824b5
·Samuele Kaplun on May 14 2008, 17:49.
Fixed mkstemp calls that leaved open files.
6d17ad8b673e
·Samuele Kaplun on May 14 2008, 16:42.

May 13 2008

Added debugging code WRT encoding, just as for the bibrank word indexer. Now…
a3b4bad725ec
·Tibor Simko on May 13 2008, 17:24.

May 12 2008

Trying to fixing select/update vs select/insert bug.
ffb2b3a00592
·Samuele Kaplun on May 12 2008, 22:00.

May 11 2008

get_word_tables now returns a list of tuples (index_id, index_tags) instead of…
8e489b9e2ffd
·Samuele Kaplun on May 11 2008, 17:35.

Apr 3 2008

Deployed task_sleep_now_if_required to bibindex.
02c67d5a4a19
·Samuele Kaplun on Apr 3 2008, 16:34.

Apr 1 2008

New cleaner signal handling. By using task_sleep_now_if_required(can_stop_too)…
cc5418ab136d
·Samuele Kaplun on Apr 1 2008, 18:59.
Fixed bibindex --reindex to work when no indexes are specified (it reindexes…
3d880f427bbe
·Samuele Kaplun on Apr 1 2008, 10:37.

Mar 31 2008

New bibsched_low_level_task_submission for enqueing tasks via API. (changes a…
0fb4ae583079
·Samuele Kaplun on Mar 31 2008, 19:08.

Mar 26 2008

Removed last spourious printed newline.
c47f7c426b8d
·Samuele Kaplun on Mar 26 2008, 17:45.
Clened fulltext indexing logging: if extraction method does not exist for a…
91c19744a9d8
·Samuele Kaplun on Mar 26 2008, 17:44.
Removed notImplemented get_words_from_local_fulltext: it will be anyway…
caec8e4f6e4e
·Samuele Kaplun on Mar 26 2008, 15:01.

Mar 25 2008

Fixed 175 cases of bad code indentation throughout the codebase. (Please set up…
67226ac65ffd
·Tibor Simko on Mar 25 2008, 21:53.

Mar 12 2008

Updated codebase to use CFG_SITE_URL instead of weburl everywhere. Updated…
2b99705c3821
·Tibor Simko on Mar 12 2008, 17:47.
Hopefully fixed "[must be sys.stdout or sys.stderr] spurious messages.
a0235b21a588
·Samuele Kaplun on Mar 12 2008, 14:32.

Mar 11 2008

Fixed verbosity of some necessary output messages.
8b53ea007afc
·Tibor Simko on Mar 11 2008, 22:37.
Different small fixes after pylint.
b6d3dcd69474
·Samuele Kaplun on Mar 11 2008, 17:48.

Mar 10 2008

Updated conf files. Replaced old style variable names (e.g. WEBURL) with new…
6dc20e3ca587
·Tibor Simko on Mar 10 2008, 20:32.

Feb 28 2008

Fixed bug about last_update_time of an index no more updated.
0035392b57e0
·Samuele Kaplun on Feb 28 2008, 10:25.

Feb 5 2008

Disabled idxPHRASE indexing until websearch will exploit it.
abe7f722fb22
·Samuele Kaplun on Feb 5 2008, 14:53.

Feb 4 2008

Updated copyright years.
eceb995a1b50
·Tibor Simko on Feb 4 2008, 18:38.

Jan 18 2008

Made washing of terms configurable. idxPHRASEs should infact not wash terms (i.
c555c383303e
·Samuele Kaplun on Jan 18 2008, 11:41.

Jan 15 2008

Implemented idxPHRASE indexes that should help when searching for exact phrases…
22223b2f9e3d
·Samuele Kaplun on Jan 15 2008, 18:14.

Jan 10 2008

Moved stemming support from static index config variable structure to column…
0affefbcb29c
·Samuele Kaplun on Jan 10 2008, 17:47.

Dec 7 2007

Fixed stemming applyed to all the indexes, by adding…
229092586df1
·Samuele Kaplun on Dec 7 2007, 18:41.

Dec 4 2007

Ported to bibdocfile, implemented -R,--reindex feature, directly reading of…
17c7f225d6e1
·Samuele Kaplun on Dec 4 2007, 10:36.

Nov 6 2007

Get rid of dbquery's escape_string() and cStringIO factory, escaping SQL…
1115e53b3879
·Tibor Simko on Nov 6 2007, 19:48.
Fixed truncation of 50+ bytes long index terms, respecting strictly UTF-8…
e72ca5f81610
·Tibor Simko on Nov 6 2007, 11:50.

Oct 18 2007

Fixed syntax error in the fulltext indexing branch.
3bbb7e469769
·Tibor Simko on Oct 18 2007, 14:56.

Oct 16 2007

Make HTM and HTML files also fulltext-indexable. This is needed for articles…
6048dd37581e
·Tibor Simko on Oct 16 2007, 16:39.

Aug 9 2007

New intbitset with universe support.
cfdd1ea34eed
·Samuele Kaplun on Aug 9 2007, 16:02.

Aug 7 2007

Correctly supporting -V flag in bibtasks.
602cc38d3271
·Samuele Kaplun on Aug 7 2007, 16:29.

Aug 3 2007

Friday commit: removed dependency to Numeric. Migrated HitSet & Co. to…
7a3572e4228f
·Samuele Kaplun on Aug 3 2007, 19:20.

Jun 8 2007

Refactored bibupload, bibindex and bibreformat tasks to new bibtask cleaned…
7b2f8868dcd2
·Samuele Kaplun on Jun 8 2007, 13:31.

Jun 6 2007

Ported to new BibTask class
c9303324e339
·Samuele Kaplun on Jun 6 2007, 19:53.

May 22 2007

Moved from the obsolete sre module to re.
9657e2ace771
·Samuele Kaplun on May 22 2007, 11:54.

May 15 2007

Clarified emergency output messages in case unrepairable errors were found…
db71f1e58654
·Tibor Simko on May 15 2007, 12:39.

Mar 27 2007

Handle UnicodeEncodeError related problems when printing messages in…
f5dd8d538885
·Tibor Simko on Mar 27 2007, 17:50.

Feb 14 2007

Updated copyright years (2007).
df75ba0b1feb
·Tibor Simko on Feb 14 2007, 19:31.

Nov 1 2006

When authentifying user-supplied email or nickname, use two SQL queries rather…
7cbd91d4ea9b
·Tibor Simko on Nov 1 2006, 15:38.

Sep 20 2006

Fixed config wildcard import everywhere. Fixed some randomly spotted import…
af4a85dac604
·Tibor Simko on Sep 20 2006, 19:27.

Sep 18 2006

Harmonized CFG_PATH_* and CFG_WEBSEARCH_* config variable names.
ec761f4dff4b
·Tibor Simko on Sep 18 2006, 12:15.

Sep 15 2006

Renamed cfg_bibindex_* config variables to follow the uppercase model.
770420bc2854
·Tibor Simko on Sep 15 2006, 11:57.

Sep 13 2006

Fixed the kwalitee problem of re-exposing some of the global config variables…
2311f7764a03
·Tibor Simko on Sep 13 2006, 16:55.
Fixed the kwalitee problem of re-exposure of some of the global config…
0674ca3c5a30
·Tibor Simko on Sep 13 2006, 16:30.

Aug 29 2006

Used urllib2 instead of urllib, mkstemp() instead of mktemp(), and removed some…
dbc8be0c4e5d
·Tibor Simko on Aug 29 2006, 21:38.
Updated every BibSched task to follow the BibTaskEx example of handling task…
536257883e68
·Tibor Simko on Aug 29 2006, 11:26.

Aug 9 2006

Fixed a bug relating to the indexation of fulltexts: When the contents of a PDF…
31f485a1e848
·Nicholas Robinson on Aug 9 2006, 15:26.

Jun 29 2006

BibSched task authentication now accepts nicknames as usernames too. Modified…
5aac7386e47f
·Tibor Simko on Jun 29 2006, 09:51.

Jun 20 2006

Changed database access code in order not to depend on MySQLdb but rather on…
1fe7d8836f40
·Tibor Simko on Jun 20 2006, 11:19.

May 4 2006

Implemented name change CDSware to CDS Invenio. Also, introduced new configure…
18b100fe8575
·Tibor Simko on May 4 2006, 16:34.

May 2 2006

Updated copyright years.
d5b36a574e3c
·Tibor Simko on May 2 2006, 14:54.

Dec 20 2005

Python imports are now done in an absolute way (from cdsware.foo import bar)…
6b1700acdd20
·Tibor Simko on Dec 20 2005, 16:39.

Dec 5 2005

Quick fix in order to be able to fulltext-index Indico URLs that do not contain…
561825a6b79f
·Tibor Simko on Dec 5 2005, 14:49.

May 12 2005

Getting rid of WML.
d41001de533e
·Tibor Simko on May 12 2005, 18:09.

Mar 29 2005

Fixed add_recIDs_by_date() in case user wants to index records based on…
e694f41220b7
·Tibor Simko on Mar 29 2005, 18:10.

Jan 6 2005

Updated copyright years.
84b18f1b27c5
·Tibor Simko on Jan 6 2005, 17:41.
Fixed a bug when calculating index id from tablename for indexes greater than…
b2dc486ac9ea
·Tibor Simko on Jan 6 2005, 10:33.