Diffusion invenio-infoscience (master) History

R3600/modules/bibupload
master069ab11497d9c2adc1357ce…

Jun 23 2008

BibDocFile+FFT to correctly support URL encoding (for spaces etc.)
170f2ea119d9
·Samuele Kaplun on Jun 23 2008, 14:28.

Jun 20 2008

New FIX-MARC (+ FIX as become FIX-ALL) directive to bibdocfilecli and FFT, used…
c7ca1047c5ad
·Samuele Kaplun on Jun 20 2008, 12:32.

Jun 19 2008

Improved FFT documentation by adding DELETE-FILE directive and some requirement…
26625dd8cfd4
·Samuele Kaplun on Jun 19 2008, 08:58.
Completed support for DELETE-FILE FFT directive.
28752c24a040
·Samuele Kaplun on Jun 19 2008, 08:53.

Jun 18 2008

When using FFT to rename a docname, take care of renaming the corresponding…
01710b6bb530
·Samuele Kaplun on Jun 18 2008, 18:05.

May 15 2008

Fixed fix call.
1e74a99408ee
·Samuele Kaplun on May 15 2008, 10:16.

May 13 2008

Changed socket default timeout from 10s to 40s. This could maybe help in…
2c9671cc8f33
·Samuele Kaplun on May 13 2008, 23:21.

Apr 16 2008

When FFT revising, if restriction/comment/description/icon is not specified, it…
b4895deb2d5b
·Samuele Kaplun on Apr 16 2008, 13:46.

Apr 15 2008

Mega commit that includes: Added support for keeping comments and descriptions…
9840677d2f4a
·Samuele Kaplun on Apr 15 2008, 16:02.
Corrected bad indentation previously committed due to misconfiguration of…
e7f7419ef6b2
·Samuele Kaplun on Apr 15 2008, 11:09.
Deployed fix_argv_paths to fix relative path issues on the command line.
8cc3b43a8385
·Samuele Kaplun on Apr 15 2008, 10:39.

Apr 7 2008

Moved some key functions out to bibdocfile. Implemented failing recid on first…
17044e7b5b7b
·Samuele Kaplun on Apr 7 2008, 17:39.
Moved CFG_BIBUPLOAD_FFT_ALLOWED_LOCAL_PATHS from bibupload_config to invenio.
c5e789cc70a2
·Samuele Kaplun on Apr 7 2008, 17:20.

Apr 3 2008

Added timeout to socket operations (e.g. urlopen) in order not to possibly hang…
614dabc993b4
·Samuele Kaplun on Apr 3 2008, 16:56.
Deployed task_sleep_now_if_required in bibupload.
c7a2cb2becc8
·Samuele Kaplun on Apr 3 2008, 13:45.

Mar 31 2008

Fixed _options & _task_params not declared.
bb8b59146af7
·Samuele Kaplun on Mar 31 2008, 21:15.
New bibsched_low_level_task_submission for enqueing tasks via API. (changes a…
0fb4ae583079
·Samuele Kaplun on Mar 31 2008, 19:08.

Mar 27 2008

Moved globally interesting CFG variables from local foo_config.py files into…
b00299effbda
·Tibor Simko on Mar 27 2008, 00:26.

Mar 25 2008

Adapted Makefiles to the new standard webdoc source directory…
e5cf1e00e21a
·Tibor Simko on Mar 25 2008, 16:09.

Mar 23 2008

Standardized unit and regression test suite building and running techniques.
3d7e61d5dbcb
·Tibor Simko on Mar 23 2008, 17:15.

Mar 20 2008

Strengthened FFT error checking. Improved regression tests.
91d7d8661bc9
·Samuele Kaplun on Mar 20 2008, 12:30.
Moved (-s, --stage) option to (-S --stage) because of clashing with (-s…
f98123ae167b
·Samuele Kaplun on Mar 20 2008, 10:23.
Added regression test to test for FFT writing rights and made regression tests…
994daaa6d7b9
·Samuele Kaplun on Mar 20 2008, 09:47.

Mar 14 2008

Restored original behaviour of storing current time as last_update time when…
86a7c9bdabe1
·Samuele Kaplun on Mar 14 2008, 19:21.
Completed bibupload support for $d in FMT, plugged check in a test case.
c118ff633251
·Samuele Kaplun on Mar 14 2008, 15:37.
Added d field to FMT tag to provide modification_date to bibupload.
a16b57aed69d
·Samuele Kaplun on Mar 14 2008, 15:10.

Mar 12 2008

Updated codebase to use CFG_SITE_URL instead of weburl everywhere. Updated…
2b99705c3821
·Tibor Simko on Mar 12 2008, 17:47.

Mar 11 2008

Phased out legacy variables 'accessurl', 'urlpath', 'images', 'htdocsurl' used…
fe3f00b1b042
·Tibor Simko on Mar 11 2008, 19:33.
Fixed <br> in *.webdoc.
f594d24fc1bb
·Samuele Kaplun on Mar 11 2008, 16:30.

Mar 10 2008

Updated conf files. Replaced old style variable names (e.g. WEBURL) with new…
6dc20e3ca587
·Tibor Simko on Mar 10 2008, 20:32.

Mar 7 2008

Fixed lowering of format bug. Now a file with extension .ps.Z can be happily…
ceb60d4f9bd2
·Samuele Kaplun on Mar 7 2008, 13:36.
Fixed bug that prevented to correctly append documents via FFT.
f5d952728470
·Samuele Kaplun on Mar 7 2008, 11:08.

Feb 14 2008

Fixed doc concerning REVERT functionality.
f69eed335c44
·Tibor Simko on Feb 14 2008, 18:36.
Formalized __str__() method of BibRecDocs, BibDoc, BibDocFile. Fixed sorting of…
6538e4e25bad
·Samuele Kaplun on Feb 14 2008, 16:01.

Feb 13 2008

Added by mistake to repository. Now removed.
6921f71419b0
·Samuele Kaplun on Feb 13 2008, 19:21.
Changed behaviour of Bibupload WRT FFT when --replace is used. It now behaves…
013cf6ac8e3f
·Samuele Kaplun on Feb 13 2008, 19:21.

Feb 4 2008

Updated copyright years.
eceb995a1b50
·Tibor Simko on Feb 4 2008, 18:38.

Jan 17 2008

Fixed archive_marcxml_for_history() for cases when the upload is being done not…
ecdbd413672d
·Tibor Simko on Jan 17 2008, 12:28.

Jan 16 2008

Fixed couple of typos in FFT guide.
699457aed413
·Samuele Kaplun on Jan 16 2008, 15:23.

Jan 9 2008

Activated writing of MARCXML versions to hstRECORD.
b544aae8d91b
·Tibor Simko on Jan 9 2008, 18:17.

Dec 20 2007

Cosmetic megacommit to standardize whitespaces in WebDoc tags ("<!-- WebDoc-Foo…
29dd5c456425
·Tibor Simko on Dec 20 2007, 14:52.
Removed WebDoc-Page-Navbar-Name and WebDoc-Page-Navbar-Select.
9ead20fd440e
·Jerome Caffaro on Dec 20 2007, 13:43.

Dec 13 2007

Phased out BibUpload Admin static intro page.
7ffbb0873cf4
·Tibor Simko on Dec 13 2007, 12:00.

Dec 7 2007

Beautified variable names.
351c7f61401f
·Samuele Kaplun on Dec 7 2007, 18:36.

Dec 6 2007

New bibdocfile library for handling fulltextes, to be used in place of file.py.
1b0dac2bf837
·Samuele Kaplun on Dec 6 2007, 17:33.
Verified webdoc files, removed WML files.
f7ea1d7731c0
·Tibor Simko on Dec 6 2007, 09:35.

Nov 30 2007

1) Bibupload FFT to support bibdocfile, 2) bibupload FFT to support icon…
3d8dbbfdba21
·Samuele Kaplun on Nov 30 2007, 17:47.

Nov 14 2007

WebDocified admin guide.
8827cfb6b2d4
·Jerome Caffaro on Nov 14 2007, 14:15.

Nov 13 2007

New bunch of regression tests for FFT.
c388f0ba6cab
·Samuele Kaplun on Nov 13 2007, 18:49.
Small bug-fixes for FFT and added verbose mode.
60b52cc20f11
·Samuele Kaplun on Nov 13 2007, 18:49.

Nov 12 2007

Added new FFT regression tests.
d77bfbec6c35
·Samuele Kaplun on Nov 12 2007, 18:07.
Trying to fix some bugs with FFTs and added support for wiping out all linked…
7da63e726510
·Samuele Kaplun on Nov 12 2007, 18:06.

Nov 8 2007

Deleted unused find_record_bibfmt(). Removed unused MySQLdb import. The file…
8d8837afe404
·Tibor Simko on Nov 8 2007, 14:58.

Nov 7 2007

More regression test (some to be fixed) and support for propagatin restrictions…
8f945cded480
·Samuele Kaplun on Nov 7 2007, 19:09.

Nov 6 2007

BibUpload-FFT now in a working state. Previous regression tests works, Demo…
689a847ba674
·Samuele Kaplun on Nov 6 2007, 17:57.
Documenting support for preserving previous details, for adding an ICON for…
a286e34a0d49
·Samuele Kaplun on Nov 6 2007, 17:52.

Nov 2 2007

Traditional Big Friday Commit (tm) :-) New FFT BibUpload implementation that…
803e1afe5d8d
·Samuele Kaplun on Nov 2 2007, 19:17.
Adding checks for downloaded files.
dd1ad491a9c4
·Samuele Kaplun on Nov 2 2007, 19:09.
Updated the specs by adding some new subfields.
6023d96be59d
·Samuele Kaplun on Nov 2 2007, 19:08.

Nov 1 2007

Added new section with the initial description of the FFT tag behaviour.
480398b2033b
·Tibor Simko on Nov 1 2007, 19:56.

Aug 7 2007

Correctly supporting -V flag in bibtasks.
602cc38d3271
·Samuele Kaplun on Aug 7 2007, 16:29.

Jun 8 2007

Refactored bibupload, bibindex and bibreformat tasks to new bibtask cleaned…
7b2f8868dcd2
·Samuele Kaplun on Jun 8 2007, 13:31.

Jun 5 2007

Added support for "strong tags" that can resist metadata replace mode ordered…
665a2b881a74
·Tibor Simko on Jun 5 2007, 15:11.

May 23 2007

Change in Fulltext file transfer (FFT): the files are copied instead of moved
6c3a25c4573f
·jkrause on May 23 2007, 15:14.

May 15 2007

Many changes, including: * Implementation of a firewall like role definition…
ceaf067fa2cd
·Samuele Kaplun on May 15 2007, 19:35.

Apr 26 2007

Changed CFG_BIBUPLOAD_EXTERNAL_OAIID_TAG to 035__a that is an independent field…
15dae91c9df9
·Tibor Simko on Apr 26 2007, 14:48.

Apr 2 2007

Added regression tests for external OAI ID tag matching facility.
03ff75c65958
·Tibor Simko on Apr 2 2007, 00:49.
Added support for external OAI ID tag matching.
6212408be663
·Tibor Simko on Apr 2 2007, 00:48.
Added CFG_BIBUPLOAD_EXTERNAL_OAIID_TAG, useful when records harvested by OAI…
29b14b73e03a
·Tibor Simko on Apr 2 2007, 00:47.

Mar 30 2007

Added regression tests for the FFT mode. To be improved later.
2ed4508574e4
·Tibor Simko on Mar 30 2007, 16:17.
First provisional implementation of the fulltext file transfer mode by Jan.
1c1bd3cc4ace
·Tibor Simko on Mar 30 2007, 16:16.

Feb 14 2007

Updated copyright years (2007).
df75ba0b1feb
·Tibor Simko on Feb 14 2007, 19:31.

Feb 6 2007

Added regression test for respecting uppercase/lowercase values when inserting…
9ad6a0e47b97
·Tibor Simko on Feb 6 2007, 19:05.
When inserting into bibxxx tables, during the look up of already existing…
04329f860a9e
·Tibor Simko on Feb 6 2007, 19:04.

Jan 26 2007

Use bibformat.format_record() rather than search_engine.print_record() to…
0c7f6ad29216
·Tibor Simko on Jan 26 2007, 10:20.

Jan 25 2007

Enhanced the complete regression test suite in order to check for the tag 001…
3d04df3b1a0b
·Tibor Simko on Jan 25 2007, 20:01.
When matching MARCXML input file records against the database, if a match was…
eb46cb3d4347
·Tibor Simko on Jan 25 2007, 19:58.

Dec 21 2006

Write a short information on the input file and the selected input mode into…
29281729a554
·Tibor Simko on Dec 21 2006, 15:31.
Fixed bad indentation.
85316c5e8c11
·Tibor Simko on Dec 21 2006, 09:52.

Dec 20 2006

When some records could not have been uploaded due to errors such as non…
32b3eb7381d7
·Tibor Simko on Dec 20 2006, 15:03.

Dec 18 2006

Fixed correct mode so that only the field instances matching the complete…
19c6e1c50d9a
·Tibor Simko on Dec 18 2006, 23:30.
Enhanced correct mode test.
bfbe710de95d
·Tibor Simko on Dec 18 2006, 23:13.
Added test on the indicator behaviour (empty indicators, blank space…
581ef364d0c1
·Tibor Simko on Dec 18 2006, 17:41.
Whenever a blank space indicator (or an old-style "empty" indicator) is…
c12e9e381d71
·Tibor Simko on Dec 18 2006, 14:59.

Dec 13 2006

All the regression tests updated to test also HTML MARC output, not only XML…
5d5f01b44d88
·Tibor Simko on Dec 13 2006, 16:45.
Fixed "reference" upload mode to populate bibxxx tables too. bibupload() now…
0243ff56930f
·Tibor Simko on Dec 13 2006, 16:44.

Dec 7 2006

Changed bibupload behaviour so that it would not modify the "mode" option from…
e790dadaddfa
·Tibor Simko on Dec 7 2006, 16:42.
Do not use table aliases in "DELETE x FROM ... LEFT JOIN" SQL statements…
b87c8d0d9798
·Tibor Simko on Dec 7 2006, 14:23.
Save option insert/replace/both (-r -i) before the upload of each record in…
d99743b653f6
·Jerome Caffaro on Dec 7 2006, 12:32.

Dec 5 2006

Adapted to the new "replace_or_insert" mode.
5d2b7d9f78c5
·Tibor Simko on Dec 5 2006, 19:38.
Fixed behaviour of CLI options; no options have values except "stage"…
1743992f0cfb
·Tibor Simko on Dec 5 2006, 19:38.

Nov 30 2006

Added regression tests for uploading of records having external SYSNO or OAI ID…
e9d1b92c399c
·Tibor Simko on Nov 30 2006, 12:17.
Added support for intelligent upload of records having external SYSNO or OAI ID…
8c20155e33a8
·Tibor Simko on Nov 30 2006, 12:15.
Use full location of external SYSNO tag, including indicators and subfield code…
fbcee3309eb7
·Tibor Simko on Nov 30 2006, 12:12.

Nov 28 2006

When comparing to None, do not use "== None" or "!= None", but rather "is None"…
5dda83af3d76
·Tibor Simko on Nov 28 2006, 18:46.
Aesthetic update of the format test values.
e355caf285a4
·Tibor Simko on Nov 28 2006, 11:44.
Corrected references tag (999, not 909).
f3af58505dcd
·Tibor Simko on Nov 28 2006, 10:46.

Nov 15 2006

Fixed some typos.
895f632bb2a4
·Jerome Caffaro on Nov 15 2006, 10:00.
Fixed wrong example in bibupload admin guide.
4c6ca52b914a
·Jerome Caffaro on Nov 15 2006, 09:54.

Nov 9 2006

Use recID 3 instead of 10 for the replace tests. Also, the tests now restore…
f9b1f25016f4
·Tibor Simko on Nov 9 2006, 16:11.