Diffusion invenio-infoscience (master) History

R3600/modules/bibrank/lib/bibrank_word_indexer.py
masterc5470de6e41dfd82c199c63…

Nov 1 2010

global: update copyright years
8f0027eea7ee
·Tibor Simko on Nov 1 2010, 23:50.
global: CDS Invenio becomes Invenio
d30e56fa001a
·Tibor Simko on Nov 1 2010, 22:08.

Apr 13 2010

WebSubmit: new fulltext conversion tools
7b3804d29a49
·Samuele Kaplun on Apr 13 2010, 09:26.

Mar 2 2010

BibRank: use register_exception()
31ef72990180
·Tibor Simko on Mar 2 2010, 12:47.

Dec 8 2008

Improved index corruption emergency help messages.
042fd85e1ace
·Tibor Simko on Dec 8 2008, 18:12.

Dec 7 2008

Fixed bibrank's max_recid detection for empty site
c570f6c8f963
·Tibor Simko on Dec 7 2008, 23:38.

Oct 27 2008

Removed $Id$ from file preamble comments.
51bfbcb9fa42
·Tibor Simko on Oct 27 2008, 22:39.

Jun 23 2008

Added new checks for Overflow o ZeroDivision.
be0d27d70025
·Samuele Kaplun on Jun 23 2008, 15:52.

May 27 2008

Added skip+register_exception for OverflowError.
feb6cfe0c8bd
·Samuele Kaplun on May 27 2008, 13:27.

May 16 2008

Corrected a message.
2549a0ef7da6
·Samuele Kaplun on May 16 2008, 16:36.

May 14 2008

Added register_exception in critical calculation to isolate DivisionByZero…
a98abc44ad92
·Samuele Kaplun on May 14 2008, 19:11.

May 13 2008

Fixed debugging code.
65a6997757aa
·Tibor Simko on May 13 2008, 14:50.
Added debugging code for spotting broken utf-8 handling.
c62839fb97e0
·Samuele Kaplun on May 13 2008, 14:38.

May 12 2008

Trying to fixing select/update vs select/insert bug.
ffb2b3a00592
·Samuele Kaplun on May 12 2008, 22:00.

Apr 3 2008

Deployed task_sleep_now_if_required to bibrank.
3ef7d4e92dfc
·Samuele Kaplun on Apr 3 2008, 16:35.

Mar 11 2008

Updated codebase to use CFG_SITE_LANG and CFG_SITE_LANGS instead of cdslang and…
7768546ebf94
·Tibor Simko on Mar 11 2008, 16:08.

Mar 10 2008

Updated conf files. Replaced old style variable names (e.g. WEBURL) with new…
6dc20e3ca587
·Tibor Simko on Mar 10 2008, 20:32.

Feb 4 2008

Updated copyright years.
eceb995a1b50
·Tibor Simko on Feb 4 2008, 18:38.

Jan 10 2008

Moved stemming support from static index config variable structure to column…
0affefbcb29c
·Samuele Kaplun on Jan 10 2008, 17:47.

Dec 7 2007

Ported ranking configuration to Porter Stemming. i.e. change 'en' language to…
f667e5895557
·Samuele Kaplun on Dec 7 2007, 18:44.

Nov 8 2007

Replaced escape_string() calls.
8a80e1c4c00c
·Tibor Simko on Nov 8 2007, 17:05.

Nov 6 2007

Get rid of dbquery's escape_string() and cStringIO factory, escaping SQL…
1115e53b3879
·Tibor Simko on Nov 6 2007, 19:48.
Fixed truncation of 50+ bytes long index terms, respecting strictly UTF-8…
e72ca5f81610
·Tibor Simko on Nov 6 2007, 11:50.

Aug 10 2007

Intbitset: Renamed universe to trailing_bits, some docstrings cosmetics, some…
610dd098425b
·Samuele Kaplun on Aug 10 2007, 18:06.
Removed not needed anymore config parameter CFG_MAX_RECID.
53a96fd8f9bb
·Samuele Kaplun on Aug 10 2007, 12:05.

Aug 9 2007

New intbitset with universe support.
cfdd1ea34eed
·Samuele Kaplun on Aug 9 2007, 16:02.

Aug 3 2007

Friday commit: removed dependency to Numeric. Migrated HitSet & Co. to…
7a3572e4228f
·Samuele Kaplun on Aug 3 2007, 19:20.

Jun 20 2007

Converted bibrank to new Bibtask code, and cleaned the code here and there.
10ebb663efaf
·Samuele Kaplun on Jun 20 2007, 19:21.

May 22 2007

Moved from the obsolete sre module to re.
9657e2ace771
·Samuele Kaplun on May 22 2007, 11:54.

May 15 2007

Adapted to the new Porter stemmer. Also, deleted trailing spaces.
23e885bd5943
·Tibor Simko on May 15 2007, 15:57.

Mar 27 2007

Handle UnicodeEncodeError related problems when printing messages in…
f5dd8d538885
·Tibor Simko on Mar 27 2007, 17:50.

Feb 14 2007

Updated copyright years (2007).
df75ba0b1feb
·Tibor Simko on Feb 14 2007, 19:31.

Dec 4 2006

Fixed free variable problems (get_date).
8e7fb32e3762
·Tibor Simko on Dec 4 2006, 14:28.

Sep 20 2006

Removed wildcard config import and fixed some randomly spotted basic kwalitee…
9522de4fc780
·Tibor Simko on Sep 20 2006, 18:54.

Sep 18 2006

Harmonized CFG_PATH_* and CFG_WEBSEARCH_* config variable names.
ec761f4dff4b
·Tibor Simko on Sep 18 2006, 12:15.

Sep 13 2006

Removed __version__ in profit of __revision__.
fcef3232f08d
·Tibor Simko on Sep 13 2006, 20:32.
Fixed the kwalitee problem of re-exposing some of the global config variables…
2311f7764a03
·Tibor Simko on Sep 13 2006, 16:55.
Fixed the kwalitee problem of re-exposure of some of the global config…
0674ca3c5a30
·Tibor Simko on Sep 13 2006, 16:30.

Aug 29 2006

Updated every BibSched task to follow the BibTaskEx example of handling task…
536257883e68
·Tibor Simko on Aug 29 2006, 11:26.

Jun 20 2006

Changed database access code in order not to depend on MySQLdb but rather on…
1fe7d8836f40
·Tibor Simko on Jun 20 2006, 11:19.

May 4 2006

Implemented name change CDSware to CDS Invenio. Also, introduced new configure…
18b100fe8575
·Tibor Simko on May 4 2006, 16:34.

May 2 2006

Updated copyright years.
d5b36a574e3c
·Tibor Simko on May 2 2006, 14:54.

Dec 20 2005

Python imports are now done in an absolute way (from cdsware.foo import bar)…
6b1700acdd20
·Tibor Simko on Dec 20 2005, 16:39.

May 12 2005

Getting rid of WML.
f7f66751768a
·Tibor Simko on May 12 2005, 17:57.

Mar 29 2005

Fixed the bibrank -m option (add_recIDs_by_date).
be497a775e31
·Tibor Simko on Mar 29 2005, 18:23.

Jan 6 2005

Updated copyright years.
84b18f1b27c5
·Tibor Simko on Jan 6 2005, 17:41.
Delete unnecessary calculation of index id from index tablename. (Leftover from…
402ec8dfc28f
·Tibor Simko on Jan 6 2005, 10:39.

Jan 3 2005

When -R is used, the program is now properly updating all term weights..or was…
4be90ebb3e00
·Trond Aksel Myklebust on Jan 3 2005, 19:23.

Dec 15 2004

Cleaned up the import statements.
0b3e18efffde
·Tibor Simko on Dec 15 2004, 10:12.

Dec 13 2004

Reflect minor BibIndex stopwords/stemmer API change.
7cb3822113dd
·Tibor Simko on Dec 13 2004, 18:56.

Dec 8 2004

Added comments, added removal of http links when indexing
5df0d0ee416e
·Trond Aksel Myklebust on Dec 8 2004, 18:47.

Nov 26 2004

Fixed some code when not doing an full update
10ae41aa58da
·Trond Aksel Myklebust on Nov 26 2004, 17:03.

Nov 23 2004

Fixed a problem with mod_python crashing
08ec71b289f2
·Trond Aksel Myklebust on Nov 23 2004, 15:12.

Nov 19 2004

Changed variables in preference
486543fceae3
·Trond Aksel Myklebust on Nov 19 2004, 14:49.
*Using stopword/stemming from /bibindex/lib and stemming/stopword functions
fb5cea213800
·Trond Aksel Myklebust on Nov 19 2004, 12:21.

Oct 12 2004

Fixed problem in word indexer: If a document already in the table was reindexed…
514d141a6055
·Trond Aksel Myklebust on Oct 12 2004, 10:50.

Oct 4 2004

Changed structure of dictionary used by singletagrankmethod,citations and…
03ef8c7918c8
·Trond Aksel Myklebust on Oct 4 2004, 11:35.

Sep 23 2004

-Modified error messages from webaccess -and something else...
6d8bfa2998fb
·Trond Aksel Myklebust on Sep 23 2004, 14:18.

Sep 21 2004

Changed error message when Psyco is not installed from "ERROR:" to "Warning:"
88850ab8f94b
·Trond Aksel Myklebust on Sep 21 2004, 17:00.

Sep 14 2004

*** empty log message ***
f1b7ee19a3cd
·Trond Aksel Myklebust on Sep 14 2004, 15:51.

Sep 7 2004

Changed a function name from word_frequency to word_similarity
9374eb47c3f2
·Trond Aksel Myklebust on Sep 7 2004, 18:09.

Sep 6 2004

-Added method for merging the result of various rank methods into one. -Removed…
675c3b8b0030
·Trond Aksel Myklebust on Sep 6 2004, 17:02.

Sep 3 2004

Modified debug information
8039cdc062ee
·Trond Aksel Myklebust on Sep 3 2004, 15:23.
-Bibrank_word_indexer: fixed bug when verbose > 0 -bibrank: fixed default…
e98f0f0028f7
·Trond Aksel Myklebust on Sep 3 2004, 10:31.

Sep 1 2004

-Modified interfaces mostly
4d1b8ae649c3
·Trond Aksel Myklebust on Sep 1 2004, 17:05.

Aug 27 2004

*Changed output from rank_records in bibrank_record_sorter, now returning (list…
a44beb8004f7
·Trond Aksel Myklebust on Aug 27 2004, 15:37.

Aug 26 2004

Modified update of last_update field in rnkMETHOD
247007d9810b
·Trond Aksel Myklebust on Aug 26 2004, 16:43.
*Fixed bug causing nothing to happen if executed before searching worked
75c14ef84ff0
·Trond Aksel Myklebust on Aug 26 2004, 16:18.
Missed an = in a if bla == bla line
5d58f1e52c40
·Trond Aksel Myklebust on Aug 26 2004, 15:56.
Modified error message when PyStemmer is not installed, or language is not…
7b1616786591
·Trond Aksel Myklebust on Aug 26 2004, 15:54.
Modified ranking files, mostly small changes in naming of methods.
945c20e71ed2
·Trond Aksel Myklebust on Aug 26 2004, 15:22.

Aug 25 2004

Methods for ranking/sorting records based on various methods like word…
b55f5a594ea0
·Trond Aksel Myklebust on Aug 25 2004, 11:05.