Diffusion invenio-infoscience (master) History

R3600/modules/bibharvest/lib
master50a211189b9fc6758f487b8…

Sep 9 2008

Fixed bug that would prevent harvesting of multiple repositories at once (file…
50a211189b9f
·Jerome Caffaro on Sep 9 2008, 08:05.

Aug 29 2008

Holding pen prototype
f34700ca03ee
·Piotr Praczyk on Aug 29 2008, 17:32.

Aug 22 2008

New page for viewing the bibupload logs (.err and .log files in the logs…
e1ccbf8648dd
·Piotr Praczyk on Aug 22 2008, 14:16.

Aug 21 2008

New page for viewing the individual record harvesting history
e43ee67de005
·Piotr Praczyk on Aug 21 2008, 18:57.
The web interface displays all the logged information now
6a349a656c35
·Piotr Praczyk on Aug 21 2008, 16:28.
Added logging when harvesting record and later (when inserting into teh…
2bd13ab78c6e
·Piotr Praczyk on Aug 21 2008, 11:52.

Aug 19 2008

New documentation sections Menu at the top of pages Bug fixes: - Little…
0d119a739a0f
·Piotr Praczyk on Aug 19 2008, 17:31.

Aug 18 2008

New page for displaying daily harvesting logs - displaying the results in…
991b5b87b3fa
·Piotr Praczyk on Aug 18 2008, 19:03.

Aug 15 2008

When testing harvesting source, record should be treated as not existing in the…
7e0be2e93271
·Piotr Praczyk on Aug 15 2008, 16:12.
Little changes in SQL tables connected with logging
349d2e584685
·Piotr Praczyk on Aug 15 2008, 13:59.

Aug 11 2008

New interface for viewing the harvesting history ( splited into months/days ...
6a21a6e38585
·Piotr Praczyk on Aug 11 2008, 18:06.

Aug 8 2008

When checking input dates, check for complete string matching, not only…
6c4fca06bb07
·Jerome Caffaro on Aug 8 2008, 13:39.

Aug 7 2008

Logging of harvested records ( new parameter for bibupload indicating the id of…
ea500c914561
·Piotr Praczyk on Aug 7 2008, 17:32.

Aug 4 2008

Do not raise an exception when trying to clean cache but file is removed in…
826eb1ff1b8b
·Jerome Caffaro on Aug 4 2008, 13:42.

Jul 25 2008

Seed of viewing the harvesting history for a given source
6d5437d0aed1
·Piotr Praczyk on Jul 25 2008, 14:13.

Jul 8 2008

Added example to help.
8c9f14c99796
·Jerome Caffaro on Jul 8 2008, 08:08.

Jul 4 2008

/tmp/cvs5L0Yky
21985bd499a8
·Piotr Praczyk on Jul 4 2008, 16:14.

Jun 27 2008

Do not insert new source if it is missing some mandatory settings.
06cbd97af879
·Jerome Caffaro on Jun 27 2008, 15:26.

Jun 6 2008

Updated status returned by task. Improved error checking.
469cbafea71f
·Jerome Caffaro on Jun 6 2008, 11:53.

May 23 2008

Added -u oaiharvest to bibupload calls.
f9806449bad3
·Samuele Kaplun on May 23 2008, 18:22.

May 19 2008

Display example of input value for "Collection(s)" field.
56b720d53005
·Jerome Caffaro on May 19 2008, 08:00.

May 15 2008

oaiarchive to specify user oaiarchive when launching bibupload.
9b80885fc295
·Samuele Kaplun on May 15 2008, 00:11.

May 12 2008

Remove ln parameter from input form in any case (webinterface_handler is adding…
4541618d8ac2
·Jerome Caffaro on May 12 2008, 15:54.

Apr 7 2008

Updating BibSched task progress.
971e8551bfc8
·Jerome Caffaro on Apr 7 2008, 14:05.
Fixed issue with retrieval of cli parameter '--repository'. Do not bibupload…
830c36a1c797
·Jerome Caffaro on Apr 7 2008, 09:50.

Apr 4 2008

Fixed an SQL INSERT statement.
f7bc986ed738
·Jerome Caffaro on Apr 4 2008, 15:32.
Deployed task_sleep_now_if_required to bibharvest suite.
4c475a4f0a85
·Samuele Kaplun on Apr 4 2008, 11:02.

Mar 31 2008

New bibsched_low_level_task_submission for enqueing tasks via API. (changes a…
0fb4ae583079
·Samuele Kaplun on Mar 31 2008, 19:08.

Mar 26 2008

Don't automatically add 'ln' parameter to URIs specified in…
adfe82ceeee4
·Jerome Caffaro on Mar 26 2008, 13:45.
Deleted trailing whitespace in all Python files.
4bc4c73928de
·Tibor Simko on Mar 26 2008, 09:25.

Mar 25 2008

Raise a 'badArgument' error when the request includes repeated arguments…
dafb9c022092
·Jerome Caffaro on Mar 25 2008, 10:32.

Mar 23 2008

Standardized unit and regression test suite building and running techniques.
3d7e61d5dbcb
·Tibor Simko on Mar 23 2008, 17:15.

Mar 12 2008

Updated codebase to use CFG_SITE_URL instead of weburl everywhere. Updated…
2b99705c3821
·Tibor Simko on Mar 12 2008, 17:47.

Mar 11 2008

Updated codebase to use CFG_SITE_SUPPORT_EMAIL instead of supportemail…
002dd06d18ec
·Tibor Simko on Mar 11 2008, 18:17.
Updated codebase to use CFG_SITE_ADMIN_EMAIL instead of adminemail everywhere.
72c35da585a5
·Tibor Simko on Mar 11 2008, 17:36.
Updated codebase to use CFG_SITE_LANG and CFG_SITE_LANGS instead of cdslang and…
7768546ebf94
·Tibor Simko on Mar 11 2008, 16:08.
Updated codebase to use CFG_SITE_NAME and CFG_SITE_NAME_INTL instead of cdsname…
ca990a4e2355
·Tibor Simko on Mar 11 2008, 12:30.

Mar 10 2008

Updated conf files. Replaced old style variable names (e.g. WEBURL) with new…
6dc20e3ca587
·Tibor Simko on Mar 10 2008, 20:32.

Mar 5 2008

Removed fragile SQL code. Thanks to Gregory Favre (EPFL).
e06bba2c3c05
·Jerome Caffaro on Mar 5 2008, 07:21.

Feb 8 2008

Better checking of admin guides availability.
ca75cb18bbaf
·Jerome Caffaro on Feb 8 2008, 08:44.

Feb 7 2008

Fixed regression tests to check for new webdoc pages.
c782c5bb2544
·Jerome Caffaro on Feb 7 2008, 08:16.

Feb 4 2008

Updated copyright years.
eceb995a1b50
·Tibor Simko on Feb 4 2008, 18:38.

Jan 9 2008

Better checking of input parameters. Fixed related test cases. Fixed bug that…
98f5faed61fb
·Jerome Caffaro on Jan 9 2008, 08:18.

Dec 19 2007

Adapted tests for default records of Atlantis.
ff386dbd802f
·Jerome Caffaro on Dec 19 2007, 15:43.

Dec 17 2007

Retrieve repository size using "exact phrase" mode.
26ccb9411d6c
·Jerome Caffaro on Dec 17 2007, 14:35.
Fixed response header of 'GetRecord' verb. Selective harvesting now use…
4801840da61b
·Jerome Caffaro on Dec 17 2007, 14:32.
Alternative Patterns mode off.
095b6dbc4e9e
·Jerome Caffaro on Dec 17 2007, 13:28.
Fixed typo.
410a53801c45
·Jerome Caffaro on Dec 17 2007, 09:21.
Added tests for selective harvesting ('from', 'until' and 'set' parameters).
6ecd3e20e72f
·Jerome Caffaro on Dec 17 2007, 09:17.

Dec 13 2007

Fixed problems with timezones and short ISO 8601 dates in 'from' and 'until'…
10676a48ebef
·Jerome Caffaro on Dec 13 2007, 12:01.
Removed unnecessary 'from mod_python import apache' statement.
0c182e7e08a0
·Jerome Caffaro on Dec 13 2007, 12:00.

Dec 12 2007

Phased out BibHarvest Admin static intro page.
ecdebf131fc0
·Tibor Simko on Dec 12 2007, 19:03.

Dec 7 2007

Still to be find a better place for this adderrorbox, but at least now things…
252561a6ed6f
·Samuele Kaplun on Dec 7 2007, 19:31.

Nov 9 2007

Fixed potentially dangerous SQL INSERT statement.
d712478ef3bc
·Tibor Simko on Nov 9 2007, 12:23.

Nov 8 2007

Replaced escape_string() occurrences.
0719bbf46078
·Tibor Simko on Nov 8 2007, 16:27.
Replaced escape_string() occurrences.
889389e10533
·Tibor Simko on Nov 8 2007, 16:21.

Oct 17 2007

Fixed timezone problem with 'from' and 'until' argument in OAI requests.
43d610347f22
·Jerome Caffaro on Oct 17 2007, 14:17.

Sep 11 2007

Fixed -r mode.
ad86ea0a5563
·Jerome Caffaro on Sep 11 2007, 11:25.
Improved task progress status feedback. Filter out whitespaces from -o --oaiset…
79931c8c9a2b
·Jerome Caffaro on Sep 11 2007, 10:04.

Sep 10 2007

Added support for processing several sets definitions in one row (-o --oaiset…
ca656401522e
·Jerome Caffaro on Sep 10 2007, 15:31.

Aug 14 2007

Fixed analysis of harvested data, that would trigger a warning when verb was…
fcc543cb6359
·Jerome Caffaro on Aug 14 2007, 15:05.
Fixed verb=ListIdentifiers and verb=ListRecords, when used with selective…
94727ee71418
·Jerome Caffaro on Aug 14 2007, 09:57.

Aug 10 2007

Fixed typo in code.
ce732596939e
·Jerome Caffaro on Aug 10 2007, 15:14.

Aug 7 2007

Correctly supporting -V flag in bibtasks.
602cc38d3271
·Samuele Kaplun on Aug 7 2007, 16:29.

Aug 3 2007

Friday commit: removed dependency to Numeric. Migrated HitSet & Co. to…
7a3572e4228f
·Samuele Kaplun on Aug 3 2007, 19:20.

Jun 21 2007

Converted oaiarchive to new BibTask code.
dbb88cc83e12
·Samuele Kaplun on Jun 21 2007, 14:33.
OaiHarvest converted to new Bibtask code.
c6b5e3f6184e
·Samuele Kaplun on Jun 21 2007, 11:20.
Moved bibharvest code to lib, with the usual trick, and cleaned the code.
95a7da56c00d
·Samuele Kaplun on Jun 21 2007, 10:16.

Jun 18 2007

Clarified comments and test on value.
bea9e7afbf17
·Jerome Caffaro on Jun 18 2007, 08:41.

Jun 14 2007

Generates normalized empty indicators for OAI datafields ("_" replaced by " ").
61f6d90e0b8b
·Jerome Caffaro on Jun 14 2007, 11:30.

Jun 13 2007

Alternate row background-color in admin web interface.
66e38325f0bc
·Jerome Caffaro on Jun 13 2007, 09:36.

Jun 12 2007

XHTML valid output.
ffb6c2776865
·Jerome Caffaro on Jun 12 2007, 14:43.
Added possibility to choose your operator (instead of default 'AND') when…
8b3005c1aad0
·Jerome Caffaro on Jun 12 2007, 12:37.

Jun 6 2007

Moved business code from bin/oaiarchive.in to lib/oaiarchive_engine.py.
54dcaa095fc5
·Jerome Caffaro on Jun 6 2007, 09:11.

May 31 2007

Fixed problem with retrieval of collections in setspec. Implemented new 'clean'…
13fdcb7409dd
·Jerome Caffaro on May 31 2007, 11:15.

May 22 2007

Moved from the obsolete sre module to re.
9657e2ace771
·Samuele Kaplun on May 22 2007, 11:54.

May 15 2007

Many changes, including: * Implementation of a firewall like role definition…
ceaf067fa2cd
·Samuele Kaplun on May 15 2007, 19:35.

May 3 2007

Handling of InvalidURL exception.
4b9d95b4bf22
·Jerome Caffaro on May 3 2007, 08:54.

Apr 23 2007

Added statistics on the number of records in harvested, converted, and filtered…
4a1ec9741b3f
·Tibor Simko on Apr 23 2007, 15:09.
Fixed sql query. Bug reported by Kam-ming Ku.
ed7c677150c2
·Jerome Caffaro on Apr 23 2007, 13:33.

Apr 2 2007

Fixed exit status code for call_bibupload().
fd0c36b240ce
·Tibor Simko on Apr 2 2007, 15:05.
Added support for h-c-f-u mode (harvest, convert, filter, upload), both to the…
3e019b6e7ecb
·Tibor Simko on Apr 2 2007, 13:46.
Removed deprecated stuff.
b0cf639821e5
·Tibor Simko on Apr 2 2007, 13:42.

Mar 30 2007

Removed documentation about non-existing option from oaiarchive help and guide.
b9872681b12e
·Jerome Caffaro on Mar 30 2007, 09:17.

Mar 27 2007

Handle UnicodeEncodeError related problems when printing messages in…
f5dd8d538885
·Tibor Simko on Mar 27 2007, 17:50.

Feb 27 2007

Improved handling of non-standard OAI responses from OAI sources (possibility…
08c313146223
·Jerome Caffaro on Feb 27 2007, 10:33.

Feb 23 2007

Fixed bug in web interface that would not correctly recognize selected sets to…
71c578509bd0
·Jerome Caffaro on Feb 23 2007, 10:31.
Fixed a bug that could prevent adding records to the OAI repository if they had…
83c76c589490
·Jerome Caffaro on Feb 23 2007, 07:58.

Feb 22 2007

Made use of XSL template for OAI DC output instead of hardcoded python.
48a13ad607ee
·Jerome Caffaro on Feb 22 2007, 08:58.

Feb 14 2007

Updated copyright years (2007).
df75ba0b1feb
·Tibor Simko on Feb 14 2007, 19:31.
Split one test.
df4d85de20bd
·Jerome Caffaro on Feb 14 2007, 15:11.

Feb 13 2007

Added performance test for serving speed of OAI repository.
cebcdb1be377
·Jerome Caffaro on Feb 13 2007, 14:20.
Reversed order of additional authors for DC output, in order to match other…
241afb4addb8
·Jerome Caffaro on Feb 13 2007, 11:13.

Feb 12 2007

Improved transformation of marcxml retrieved from database for oaimarcxml…
54fbac1d569d
·Jerome Caffaro on Feb 12 2007, 15:57.
Improved speed for OAI harvesting with verb=ListRecords&metadataPrefix=marcxml.
f634c6df34cf
·Jerome Caffaro on Feb 12 2007, 15:35.

Feb 9 2007

Improved speed.
8943b041e627
·Jerome Caffaro on Feb 9 2007, 15:11.

Feb 6 2007

Eliminated duplicate sets in OAI-PMH 'ListSets' response. Eliminated duplicates…
b5c8693f96cf
·Jerome Caffaro on Feb 6 2007, 15:33.
Corrected link to guide in the admin pages.
12749fbe911d
·Jerome Caffaro on Feb 6 2007, 15:29.

Jan 26 2007

Fixed   typos, thanks to Ferran Jorba.
4723496fab5a
·Tibor Simko on Jan 26 2007, 15:13.

Jan 12 2007

More detailed status returned by validate() function when validating OAI source.
1840a1f79955
·Jerome Caffaro on Jan 12 2007, 09:54.