BibAuthorID: major backend rewrite and speed ups
bd39f76ec091
Actions

Authored by Samuele Carli <samuele.carli@cern.ch> on Jan 24 2012, 17:17.

Description

BibAuthorID: major backend rewrite and speed ups

New BibAuthorID fast update and garbage collection algorithm: Rabbit. Taking advantage of the new database shape, the algorithm is much faster, reliable and performant then ever. It can be now scheduled and run ideally several times per hour, so that BibAuthorID world is always up to date with the rest of the system.

New full disambiguation algorithm: Tortoise. The new full disambiguation algorithm allows for state of the art quality, periodic full-disambiguation of authors. NB: While it is possible to use it for starting a system from scratch, the merging algorithm required for periodic updates is not fully production ready yet.

Lots of bugfixes, optimizations and improvements; the backend code have been rewritten from scratch.

Co-authored-by: Nikola Yolov <nikola.yolov@cern.ch>

Committed

Tibor Simko <tibor.simko@cern.ch>

Apr 3 2012, 16:41

Parents

Branches

Unknown

Tags

Unknown

Tibor Simko <tibor.simko@cern.ch> committed R3600:bd39f76ec091: BibAuthorID: major backend rewrite and speed ups (authored by Samuele Carli <samuele.carli@cern.ch>).Apr 3 2012, 16:41

				Path
	P			modules/bibauthorid/bin/bibauthorid.in Copied from modules/miscutil/bin/dbdump.in
	M			modules/bibauthorid/lib/Makefile.am
	M			modules/bibauthorid/lib/bibauthorid_backinterface.py
	P			modules/bibauthorid/lib/bibauthorid_cli.py Copied from modules/bibharvest/bin/oairepositoryupdater.in
	M			modules/bibauthorid/lib/bibauthorid_cluster_set.py
	M			modules/bibauthorid/lib/bibauthorid_comparison.py
	M			modules/bibauthorid/lib/bibauthorid_config.py
	M			modules/bibauthorid/lib/bibauthorid_daemon.py
	M			modules/bibauthorid/lib/bibauthorid_dbinterface.py
	M			modules/bibauthorid/lib/bibauthorid_frontinterface.py
	P			modules/bibauthorid/lib/bibauthorid_general_utils.py Copied from modules/bibauthorid/lib/bibauthorid_searchinterface.py
	A			modules/bibauthorid/lib/bibauthorid_matrix_optimization.py
	A			modules/bibauthorid/lib/bibauthorid_merge.py
	M			modules/bibauthorid/lib/bibauthorid_name_utils.py
	M			modules/bibauthorid/lib/bibauthorid_personid_maintenance.py
	M			modules/bibauthorid/lib/bibauthorid_prob_matrix.py
	A			modules/bibauthorid/lib/bibauthorid_rabbit.py
	M			modules/bibauthorid/lib/bibauthorid_string_utils.py
	M			modules/bibauthorid/lib/bibauthorid_templates.py
	A			modules/bibauthorid/lib/bibauthorid_tortoise.py
	M			modules/bibauthorid/lib/bibauthorid_webapi.py
	M			modules/bibauthorid/lib/bibauthorid_webinterface.py
	M			modules/bibauthorid/lib/bibauthorid_wedge.py
	M			modules/miscutil/sql/tabbibclean.sql
	M			modules/miscutil/sql/tabcreate.sql
	M			modules/miscutil/sql/tabdrop.sql
	M			modules/websearch/lib/websearch_webinterface.py