Homec4science

OAIHarvest: Integrate new workflows

Authored by Guillaume Lastecoueres <PX9e@gmx.fr> on Oct 30 2013, 18:42.

Description

OAIHarvest: Integrate new workflows

  • Refactor oai_harvest_daemon to use new workflows module for post process.
  • Several PEP8 fixes.

Bibworkflow: MarcXML processing workflow

  • Sub workflow exception improvement which allow to centralize all the exceptions of children workflow and to get the full stacktrace of each
  • Convert_record_to_bibfield task for workflow which convert any marcxml file to a dictionnary thanks to bibfield, the goal is to avoid the old file system based inter function communication protocol
  • Post_process after celery reception of an WorkflowObject to reattach to the new DB instance all the DB object which couled be not referenced
  • Add security of type cohesion into the WorkflowObject Transporter, it is a hot fix in wait of the database modification which allow to tranform potential string representation of dictionnary into dictionnary.
  • New workflow tasks, all compatibles with bibifield dictionnary representation fulltext_download with bibfield support, plot_extract with bibfield support, ref_extraction witht bibfield support, author_list witht bibfield support

OAIHarvest: Various import fixes and PEP8

Conflicts:
invenio/legacy/oaiharvest/daemon.py

NOTE: beware, patch amended by Tibor to use new file names

Details

Committed
kuncarJun 4 2014, 16:03
Parents
R3600:e79c44ffe373: Plotextractor: XML direct output option
Branches
Unknown
Tags
Unknown

Event Timeline

Jiri Kuncar <jiri.kuncar@cern.ch> committed R3600:b13cf437ab4d: OAIHarvest: Integrate new workflows (authored by Guillaume Lastecoueres <PX9e@gmx.fr>).Jun 4 2014, 16:03