Diffusion invenio-infoscience (legacy) History

R3600/modules/bibharvest/lib
legacy052735ae7b68ad0a60e6dbc…

Sep 29 2009

WebStyle: WSGI web interface handler
052735ae7b68
·Samuele Kaplun / Tibor Simko on Sep 29 2009, 18:31.

Jul 17 2009

BibHarvest: cleaner OAI harvester vs repository
fe19901820d8
·Jerome Caffaro / Tibor Simko on Jul 17 2009, 11:02.

Jul 15 2009

Unified use of md5.md5 vs hashlib.md5
de3d4b5c1275
·Samuele Kaplun / Tibor Simko on Jul 15 2009, 11:10.
Unified use of set vs sets.Set
f6754cfcaebd
·Samuele Kaplun / Tibor Simko on Jul 15 2009, 11:10.

Jun 26 2009

htmlutils: new remove_html_markup() API function
1c3f5175204b
·Tibor Simko on Jun 26 2009, 17:23.

Jun 15 2009

Global move from javadoc to epydoc docstring style
40394efc1646
·Tibor Simko on Jun 15 2009, 18:02.

Mar 5 2009

textutils: centralized encode_for_xml() function
c3a322f0eaf6
·Benoit Thiell / Tibor Simko on Mar 5 2009, 10:01.

Feb 11 2009

BibHarvest: timestamp harvesting test case fixed
34c9896da7a0
·Jerome Caffaro / Tibor Simko on Feb 11 2009, 11:57.

Dec 2 2008

Fix usage of tabs in code.
7ab0dcd8ea47
·Samuele Kaplun / Tibor Simko on Dec 2 2008, 16:19.

Nov 27 2008

Correctly escaping run_sql() in oai_repository.
ede559aedb86
·Samuele Kaplun / Tibor Simko on Nov 27 2008, 14:47.

Nov 24 2008

Deleted unnecessary cvsignore files.
f80152df3863
·Tibor Simko on Nov 24 2008, 17:08.

Nov 4 2008

New search_engine.get_record function.
9dc22287761b
·Samuele Kaplun on Nov 4 2008, 17:58.

Oct 29 2008

Refactored oaiarchive and simplified usage.
469c776c6f42
·Jerome Caffaro on Oct 29 2008, 08:55.

Oct 28 2008

Merge commit 'jerome/fix-oairepository-admin-interface'
d96a95c30e63
·Tibor Simko on Oct 28 2008, 11:32.
Added some tips within the interface.
71eb8c69e8d0
·Jerome Caffaro on Oct 28 2008, 10:36.
Renamed "setCollection" to "collection" to remove ambiguity.
f9f073419dd4
·Jerome Caffaro on Oct 28 2008, 09:00.
Removed unused (?) oaitable(..) function.
18c3e4f8bea0
·Jerome Caffaro on Oct 28 2008, 08:56.
Fixed addition and modifications of settings.
7a8eb43801d6
·Jerome Caffaro on Oct 28 2008, 08:48.

Oct 27 2008

Removed $Id$ from file preamble comments.
51bfbcb9fa42
·Tibor Simko on Oct 27 2008, 22:39.

Oct 23 2008

Added support for HTTPS and authentication
24445ad36851
·Jerome Caffaro on Oct 23 2008, 11:18.

Sep 26 2008

Merged task_low_level_submission and task_low_level_submission_tuple.
9a5eb5d174c1
·Samuele Kaplun on Sep 26 2008, 12:14.

Sep 9 2008

Fixed bug that would prevent harvesting of multiple repositories at once (file…
50a211189b9f
·Jerome Caffaro on Sep 9 2008, 08:05.

Aug 29 2008

Holding pen prototype
f34700ca03ee
·Piotr Praczyk on Aug 29 2008, 17:32.

Aug 22 2008

New page for viewing the bibupload logs (.err and .log files in the logs…
e1ccbf8648dd
·Piotr Praczyk on Aug 22 2008, 14:16.

Aug 21 2008

New page for viewing the individual record harvesting history
e43ee67de005
·Piotr Praczyk on Aug 21 2008, 18:57.
The web interface displays all the logged information now
6a349a656c35
·Piotr Praczyk on Aug 21 2008, 16:28.
Added logging when harvesting record and later (when inserting into teh…
2bd13ab78c6e
·Piotr Praczyk on Aug 21 2008, 11:52.

Aug 19 2008

New documentation sections Menu at the top of pages Bug fixes: - Little…
0d119a739a0f
·Piotr Praczyk on Aug 19 2008, 17:31.

Aug 18 2008

New page for displaying daily harvesting logs - displaying the results in…
991b5b87b3fa
·Piotr Praczyk on Aug 18 2008, 19:03.

Aug 15 2008

When testing harvesting source, record should be treated as not existing in the…
7e0be2e93271
·Piotr Praczyk on Aug 15 2008, 16:12.
Little changes in SQL tables connected with logging
349d2e584685
·Piotr Praczyk on Aug 15 2008, 13:59.

Aug 11 2008

New interface for viewing the harvesting history ( splited into months/days ...
6a21a6e38585
·Piotr Praczyk on Aug 11 2008, 18:06.

Aug 8 2008

When checking input dates, check for complete string matching, not only…
6c4fca06bb07
·Jerome Caffaro on Aug 8 2008, 13:39.

Aug 7 2008

Logging of harvested records ( new parameter for bibupload indicating the id of…
ea500c914561
·Piotr Praczyk on Aug 7 2008, 17:32.

Aug 4 2008

Do not raise an exception when trying to clean cache but file is removed in…
826eb1ff1b8b
·Jerome Caffaro on Aug 4 2008, 13:42.

Jul 25 2008

Seed of viewing the harvesting history for a given source
6d5437d0aed1
·Piotr Praczyk on Jul 25 2008, 14:13.

Jul 8 2008

Added example to help.
8c9f14c99796
·Jerome Caffaro on Jul 8 2008, 08:08.

Jul 4 2008

/tmp/cvs5L0Yky
21985bd499a8
·Piotr Praczyk on Jul 4 2008, 16:14.

Jun 27 2008

Do not insert new source if it is missing some mandatory settings.
06cbd97af879
·Jerome Caffaro on Jun 27 2008, 15:26.

Jun 6 2008

Updated status returned by task. Improved error checking.
469cbafea71f
·Jerome Caffaro on Jun 6 2008, 11:53.

May 23 2008

Added -u oaiharvest to bibupload calls.
f9806449bad3
·Samuele Kaplun on May 23 2008, 18:22.

May 19 2008

Display example of input value for "Collection(s)" field.
56b720d53005
·Jerome Caffaro on May 19 2008, 08:00.

May 15 2008

oaiarchive to specify user oaiarchive when launching bibupload.
9b80885fc295
·Samuele Kaplun on May 15 2008, 00:11.

May 12 2008

Remove ln parameter from input form in any case (webinterface_handler is adding…
4541618d8ac2
·Jerome Caffaro on May 12 2008, 15:54.

Apr 7 2008

Updating BibSched task progress.
971e8551bfc8
·Jerome Caffaro on Apr 7 2008, 14:05.
Fixed issue with retrieval of cli parameter '--repository'. Do not bibupload…
830c36a1c797
·Jerome Caffaro on Apr 7 2008, 09:50.

Apr 4 2008

Fixed an SQL INSERT statement.
f7bc986ed738
·Jerome Caffaro on Apr 4 2008, 15:32.
Deployed task_sleep_now_if_required to bibharvest suite.
4c475a4f0a85
·Samuele Kaplun on Apr 4 2008, 11:02.

Mar 31 2008

New bibsched_low_level_task_submission for enqueing tasks via API. (changes a…
0fb4ae583079
·Samuele Kaplun on Mar 31 2008, 19:08.

Mar 26 2008

Don't automatically add 'ln' parameter to URIs specified in…
adfe82ceeee4
·Jerome Caffaro on Mar 26 2008, 13:45.
Deleted trailing whitespace in all Python files.
4bc4c73928de
·Tibor Simko on Mar 26 2008, 09:25.

Mar 25 2008

Raise a 'badArgument' error when the request includes repeated arguments…
dafb9c022092
·Jerome Caffaro on Mar 25 2008, 10:32.

Mar 23 2008

Standardized unit and regression test suite building and running techniques.
3d7e61d5dbcb
·Tibor Simko on Mar 23 2008, 17:15.

Mar 12 2008

Updated codebase to use CFG_SITE_URL instead of weburl everywhere. Updated…
2b99705c3821
·Tibor Simko on Mar 12 2008, 17:47.

Mar 11 2008

Updated codebase to use CFG_SITE_SUPPORT_EMAIL instead of supportemail…
002dd06d18ec
·Tibor Simko on Mar 11 2008, 18:17.
Updated codebase to use CFG_SITE_ADMIN_EMAIL instead of adminemail everywhere.
72c35da585a5
·Tibor Simko on Mar 11 2008, 17:36.
Updated codebase to use CFG_SITE_LANG and CFG_SITE_LANGS instead of cdslang and…
7768546ebf94
·Tibor Simko on Mar 11 2008, 16:08.
Updated codebase to use CFG_SITE_NAME and CFG_SITE_NAME_INTL instead of cdsname…
ca990a4e2355
·Tibor Simko on Mar 11 2008, 12:30.

Mar 10 2008

Updated conf files. Replaced old style variable names (e.g. WEBURL) with new…
6dc20e3ca587
·Tibor Simko on Mar 10 2008, 20:32.

Mar 5 2008

Removed fragile SQL code. Thanks to Gregory Favre (EPFL).
e06bba2c3c05
·Jerome Caffaro on Mar 5 2008, 07:21.

Feb 8 2008

Better checking of admin guides availability.
ca75cb18bbaf
·Jerome Caffaro on Feb 8 2008, 08:44.

Feb 7 2008

Fixed regression tests to check for new webdoc pages.
c782c5bb2544
·Jerome Caffaro on Feb 7 2008, 08:16.

Feb 4 2008

Updated copyright years.
eceb995a1b50
·Tibor Simko on Feb 4 2008, 18:38.

Jan 9 2008

Better checking of input parameters. Fixed related test cases. Fixed bug that…
98f5faed61fb
·Jerome Caffaro on Jan 9 2008, 08:18.

Dec 19 2007

Adapted tests for default records of Atlantis.
ff386dbd802f
·Jerome Caffaro on Dec 19 2007, 15:43.

Dec 17 2007

Retrieve repository size using "exact phrase" mode.
26ccb9411d6c
·Jerome Caffaro on Dec 17 2007, 14:35.
Fixed response header of 'GetRecord' verb. Selective harvesting now use…
4801840da61b
·Jerome Caffaro on Dec 17 2007, 14:32.
Alternative Patterns mode off.
095b6dbc4e9e
·Jerome Caffaro on Dec 17 2007, 13:28.
Fixed typo.
410a53801c45
·Jerome Caffaro on Dec 17 2007, 09:21.
Added tests for selective harvesting ('from', 'until' and 'set' parameters).
6ecd3e20e72f
·Jerome Caffaro on Dec 17 2007, 09:17.

Dec 13 2007

Fixed problems with timezones and short ISO 8601 dates in 'from' and 'until'…
10676a48ebef
·Jerome Caffaro on Dec 13 2007, 12:01.
Removed unnecessary 'from mod_python import apache' statement.
0c182e7e08a0
·Jerome Caffaro on Dec 13 2007, 12:00.

Dec 12 2007

Phased out BibHarvest Admin static intro page.
ecdebf131fc0
·Tibor Simko on Dec 12 2007, 19:03.

Dec 7 2007

Still to be find a better place for this adderrorbox, but at least now things…
252561a6ed6f
·Samuele Kaplun on Dec 7 2007, 19:31.

Nov 9 2007

Fixed potentially dangerous SQL INSERT statement.
d712478ef3bc
·Tibor Simko on Nov 9 2007, 12:23.

Nov 8 2007

Replaced escape_string() occurrences.
0719bbf46078
·Tibor Simko on Nov 8 2007, 16:27.
Replaced escape_string() occurrences.
889389e10533
·Tibor Simko on Nov 8 2007, 16:21.

Oct 17 2007

Fixed timezone problem with 'from' and 'until' argument in OAI requests.
43d610347f22
·Jerome Caffaro on Oct 17 2007, 14:17.

Sep 11 2007

Fixed -r mode.
ad86ea0a5563
·Jerome Caffaro on Sep 11 2007, 11:25.
Improved task progress status feedback. Filter out whitespaces from -o --oaiset…
79931c8c9a2b
·Jerome Caffaro on Sep 11 2007, 10:04.

Sep 10 2007

Added support for processing several sets definitions in one row (-o --oaiset…
ca656401522e
·Jerome Caffaro on Sep 10 2007, 15:31.

Aug 14 2007

Fixed analysis of harvested data, that would trigger a warning when verb was…
fcc543cb6359
·Jerome Caffaro on Aug 14 2007, 15:05.
Fixed verb=ListIdentifiers and verb=ListRecords, when used with selective…
94727ee71418
·Jerome Caffaro on Aug 14 2007, 09:57.

Aug 10 2007

Fixed typo in code.
ce732596939e
·Jerome Caffaro on Aug 10 2007, 15:14.

Aug 7 2007

Correctly supporting -V flag in bibtasks.
602cc38d3271
·Samuele Kaplun on Aug 7 2007, 16:29.

Aug 3 2007

Friday commit: removed dependency to Numeric. Migrated HitSet & Co. to…
7a3572e4228f
·Samuele Kaplun on Aug 3 2007, 19:20.

Jun 21 2007

Converted oaiarchive to new BibTask code.
dbb88cc83e12
·Samuele Kaplun on Jun 21 2007, 14:33.
OaiHarvest converted to new Bibtask code.
c6b5e3f6184e
·Samuele Kaplun on Jun 21 2007, 11:20.
Moved bibharvest code to lib, with the usual trick, and cleaned the code.
95a7da56c00d
·Samuele Kaplun on Jun 21 2007, 10:16.

Jun 18 2007

Clarified comments and test on value.
bea9e7afbf17
·Jerome Caffaro on Jun 18 2007, 08:41.

Jun 14 2007

Generates normalized empty indicators for OAI datafields ("_" replaced by " ").
61f6d90e0b8b
·Jerome Caffaro on Jun 14 2007, 11:30.

Jun 13 2007

Alternate row background-color in admin web interface.
66e38325f0bc
·Jerome Caffaro on Jun 13 2007, 09:36.

Jun 12 2007

XHTML valid output.
ffb6c2776865
·Jerome Caffaro on Jun 12 2007, 14:43.
Added possibility to choose your operator (instead of default 'AND') when…
8b3005c1aad0
·Jerome Caffaro on Jun 12 2007, 12:37.

Jun 6 2007

Moved business code from bin/oaiarchive.in to lib/oaiarchive_engine.py.
54dcaa095fc5
·Jerome Caffaro on Jun 6 2007, 09:11.

May 31 2007

Fixed problem with retrieval of collections in setspec. Implemented new 'clean'…
13fdcb7409dd
·Jerome Caffaro on May 31 2007, 11:15.

May 22 2007

Moved from the obsolete sre module to re.
9657e2ace771
·Samuele Kaplun on May 22 2007, 11:54.

May 15 2007

Many changes, including: * Implementation of a firewall like role definition…
ceaf067fa2cd
·Samuele Kaplun on May 15 2007, 19:35.

May 3 2007

Handling of InvalidURL exception.
4b9d95b4bf22
·Jerome Caffaro on May 3 2007, 08:54.

Apr 23 2007

Added statistics on the number of records in harvested, converted, and filtered…
4a1ec9741b3f
·Tibor Simko on Apr 23 2007, 15:09.