Homec4science
Diffusion invenio-infoscience (maint-2.0) Browse

R3600/invenio/legacy/bibsched/doc/hacking/bibtask-api.webdoc
880f13550f21maint-2.0

Last Change
Raw File

bibtask-api.webdoc
View Options

# -*- mode: html; coding: utf-8; -*-
# This file is part of Invenio.
# Copyright (C) 2010, 2011 CERN.
#
# Invenio is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Invenio is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Invenio; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
<!-- WebDoc-Page-Title: BibTask -->
<!-- WebDoc-Page-Navtrail: <a class="navtrail" href="<CFG_SITE_URL>/help/hacking">Hacking Invenio</a> &gt; <a class="navtrail" href="<CFG_SITE_URL>/help/hacking/bibsched-internals">BibSched Internals</a> -->
<!-- WebDoc-Page-Revision: $Id$ -->
<h2>Introduction</h2>
<p>This is an howto create a Bibliographic Task suitable to be used with bibsched.</p>
<h2>A BibTask</h2>
<p>For starting developing a new bibtask first take a look at the source code of bibtaskex.py.</p>
<p>A BibTask "foobar" is well done when:
<ul>
<li>it calls in its main function <tt>bibsched.task_init</tt> with the proper parameters</li>
<li>it has an associated WebAccess action called "runfoobar"</li>
<li>it has an executable installed under bin directory called "foobar"</li>
<li><tt>bibtask_config.py</tt> is updated so that <tt>CFG_BIBTASK_VALID_TASKS</tt> contains "foobar"</li>
</ul>
</p>
<h3>Available API</h3>
<p>Being a bibtask, your script should use <tt>bibtask.write_message</tt> each time it needs to inform the user about something.</p>
<p>It should use <tt>bibtask.task_set_option</tt> and <tt>bibtask.task_get_option</tt> in order to interact with the bibtask options (e.g. verbosity, runtime, sleeptime, priority...)</p>
<p>It should use <tt>bibtask.task_set_task_param</tt> and <tt>bibtask.task_get_task_param</tt> in order to interact with its specific command line options.</p>
<p>It should use <tt>bibtask.task_update_progress</tt> in order to update the progress information on the task that will be displayed in bibsched.</p>
<p>It should call <tt>bibtask.task_sleep_now_if_required</tt> in each part of the code where it's safe to sleep or stop, e.g. outside of atomic operations or transactions. If not sure, just not use this function and the task won't ever be stopped or put to sleep.</p>
<p>It should implement a function to be passed to <tt>task_init</tt> via the <tt>task_submit_elaborate_specific_parameter_fnc</tt> parameter to handle specific command line parameters of the foobar bibtask.</p>
<p>It should implement a function to be passed to <tt>task_init</tt> via the <tt>task_submit_check_options_fnc</tt> parameter to have a chance to check the correctness of command line options before the task is submitted to the queue or executed.</p>
<p>It should implement a main function to be passed to <tt>task_init</tt> via the <tt>task_run_fnc</tt> parameter as the core of the bibtask.</p>
<p>It can be scheduled directly via a Python script by using the <tt>task_low_level_submission</tt> API.
<h3>Detailed API</h3>
<h4>bibtask.task_init</h4>
<pre>
def task_init(
authorization_action="",
authorization_msg="",
description="",
help_specific_usage="",
version=__revision__,
specific_params=("", []),
task_stop_helper_fnc=None,
task_submit_elaborate_specific_parameter_fnc=None,
task_submit_check_options_fnc=None,
task_run_fnc=None):
""" Initialize a BibTask.
@param authorization_action is the name of the authorization action
connected with this task;
@param authorization_msg is the header printed when asking for an
authorization password;
@param description is the generic description printed in the usage page;
@param help_specific_usage is the specific parameter help
@param task_stop_fnc is a function that will be called
whenever the task is stopped
@param task_submit_elaborate_specific_parameter_fnc will be called passing
a key and a value, for parsing specific cli parameters. Must return True if
it has recognized the parameter. Must eventually update the options with
bibtask_set_option;
@param task_submit_check_options must check the validity of options (via
bibtask_get_option) once all the options where parsed;
@param task_run_fnc will be called as the main core function. Must return
False in case of errors.
"""
</pre>
<h4>bibtask.task_{set/get/has}_option</h4>
<pre>
def task_set_option(key, value):
"""Set an value to key in the option dictionary of the task"""
</pre>
<pre>
def task_get_option(key, default=None):
"""Returns the value corresponding to key in the option dictionary of the task"""
</pre>
<pre>
def task_has_option(key):
"""Map the has_key query to _options"""
</pre>
<h4>bibtask.task_{set/get}_param</h4>
<pre>
def task_get_task_param(key, default=None):
"""Returns the value corresponding to the particular task param"""
</pre>
<pre>
def task_set_task_param(key, value):
"""Set the value corresponding to the particular task param"""
</pre>
<h4>bibtask.task_update_progress</h4>
<pre>
def task_update_progress(msg):
"""Updates progress information in the BibSched task table."""
</pre>
<h4>bibtask.write_message</h4>
<pre>
def write_message(msg, stream=sys.stdout, verbose=1):
"""Write message and flush output stream (may be sys.stdout or sys.stderr).
Useful for debugging stuff."""
</pre>
<h4>bibtask.task_sleep_now_if_required</h4>
<pre>
def task_sleep_now_if_required(can_stop_too=False):
"""This function should be called during safe state of BibTask,
e.g. after flushing caches or outside of run_sql calls.
"""
</pre>
<h4>bibtask.task_low_level_submission</h4>
<pre>
def task_low_level_submission(name, user, *argv):
"""Let special lowlevel enqueuing of a task on the bibsche queue.
@param name: is the name of the bibtask. It must be a valid executable under
C{CFG_BINDIR}.
@type name: string
@param user: is a string that will appear as the "user" submitting the task.
Since task are submitted via API it make sense to set the
user to the name of the module/function that called
task_low_level_submission.
@type user: string
@param argv: are all the additional CLI parameters that would have been
passed on the CLI (one parameter per variable).
e.g.:
>>> task_low_level_submission('bibupload', 'admin', '-a', '/tmp/z.xml')
@type: strings
@return: the task identifier when the task is correctly enqueued.
@rtype: int
@note: use absolute paths in argv
"""
</pre>
<h4>bibtask.get_modified_records_since</h4>
<pre>
def get_modified_records_since(modification_date):
"""
Return the set of modified record since the given
modification_date.
@param modification_date: Return records modified after this date
@type modification_date datetime
"""
</pre>
c4science · Help