Homec4science
Diffusion invenio-infoscience (iugw2017) Browse

R3600/invenio/legacy/bibrank/doc/admin/bibrank-admin-guide.webdoc
aa232da412f9iugw2017

Last Change
Raw File

bibrank-admin-guide.webdoc
View Options

## -*- mode: html; coding: utf-8; -*-
## This file is part of Invenio.
## Copyright (C) 2007, 2008, 2010, 2011 CERN.
##
## Invenio is free software; you can redistribute it and/or
## modify it under the terms of the GNU General Public License as
## published by the Free Software Foundation; either version 2 of the
## License, or (at your option) any later version.
##
## Invenio is distributed in the hope that it will be useful, but
## WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
## General Public License for more details.
##
## You should have received a copy of the GNU General Public License
## along with Invenio; if not, write to the Free Software Foundation, Inc.,
## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
<!-- WebDoc-Page-Title: BibRank Admin Guide -->
<!-- WebDoc-Page-Navtrail: <a class="navtrail" href="<CFG_SITE_URL>/help/admin<lang:link/>">_(Admin Area)_</a> -->
<!-- WebDoc-Page-Revision: $Id$ -->
<h2>Contents</h2>
<strong>1.<a href="#o">Overview</a></strong><br/>
<strong>2.<a href="#c">Configuration Conventions</a></strong><br/>
<strong>3.<a href="#bai">BibRank Admin Interface</a></strong><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.1.<a href="#mi">Main interface</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.2.<a href="#ar">Add rank method</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.3.<a href="#sd">Show details of rank method</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.4.<a href="#mr">Modify rank method</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.5.<a href="#dr">Delete rank method</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.6.<a href="#mt">Modify translations</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.7.<a href="#mc">Modify visibility toward collections</a><br/>
<strong>4.<a href="#bd">BibRank Daemon</a></strong><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4.1.<a href="#cli1">Command Line Interface</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4.2.<a href="#ubd">Using BibRank</a><br/>
<strong>5.<a href="#brm">BibRank Methods</a></strong><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.1.<a href="#str">Single tag rank method</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.2.<a href="#wrd">Word Similarity/Similar Records</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.2.1.<a href="#wrdsolr">Solr Word Similarity/Similar Records</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.3.<a href="#tdcc">Time-dependent citation counts</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.4.<a href="#lbr">Link-based ranking</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.5.<a href="#tdlbr">Time-dependent link-based ranking</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.6.<a href="#cmb">Combined method</a><br/>
<strong>6.<a href="#bt">bibrankgkb Tool</a></strong><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6.1.<a href="#cli2">Command Line Interface</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6.2.<a href="#ubt">Using bibrankgkb</a><br/>
<strong>7.<a href="#ainf">Additional Information</a></strong><br/>
<a name="o"></a><h2>1. Overview</h2>
<p>The bibrank module consist currently of two tools:
<br/><br/>bibrank - Generates ranking data for ranking search results based on methods like:</p>
<blockquote>
<pre>
Journal Impact Factor
Word Similarity/Similar Records
Combined Method
&nbsp;##Number of downloads
&nbsp;##Author Impact
&nbsp;##Citation Impact
</pre>
</blockquote>
<p>bibrankgkb - For generating knowledge base files for use with bibrank
</p>
<p>The bibrankgkb may not be necessary to use, it depends on which ranking methods you are planning
to use, and what data you already got. This guide will take you through the necessary steps in detail in
order to create different kinds of ranking methods for the search engine to use.
</p>
<a name="c"></a><h2>2. Configuration Conventions</h2>
<blockquote>
<pre>
- comment line starts with '#' sign in the first column
- each section in a configuration file is declared inside '[' ']' signs
- values in knowledgebasefiles are separated by '---'
</pre>
</blockquote>
<a name="bai"></a><h2>3. BibRank Admin Interface</h2>
<p>The bibrank web interface enables you to modify the configuration of most aspects of BibRank. For full functionality, it is advised to
let the http-daemon have write/read access to your invenio/etc/bibrank directory. If this is not wanted, you have to edit the configuration files from the console using your favourite text editor.</p>
<a name="mi"></a><h3>3.1 Main interface</h3>
<p>In the main interface screen, you see a list of all rank methods currently added. Each rank method is identified by the rank method code. To find out about the functionality available, check out the topics below.</p>
<b>Explanation of concepts</b>
<blockquote>
<pre>
Rank method:
A method responsible for creating the necessary data to rank a result.
Translations:
Each rank method may have many names in many languages.
Collections:
Which collections the rank method should be visible in.
</pre>
</blockquote>
<a name="ar"></a><h3>3.2 Add rank method</h3>
<p>When pressing the link in the upper right corner from the main interface, you will see the interface for adding a new rank method. The two available options that needs to be decided upon, are the bibrank code and the template to use, both values can be changed later. The bibrank code is used by the bibrank daemon to run the method, and should be fairly short without spaces. Which template you are using, decides how the ranking will be done, and must before used, be changed to suit your Invenio configuration. When confirming to add a new rank method, it will be added to the list of possible rank methods, and a configuration file will be created if the httpd user has proper rights to the 'invenio/etc/bibrank' directory. If not, the file has to manually be created with the name 'bibrankcode.cfg' where bibrankcode is the same as given in the interface.</p>
<a name="sd"></a><h3>3.3 Show details of rank method</h3>
<p>This interface gives you an overview of the current status of the rank method, and gives direct access to the various interfaces for changing the configuration.
In the overview section, you see the bibrank code, for use with the bibrank daemon, and the date for the last run of the rank method.
In the statistics section you see how many records have been added to the rank method and other statistic data. In the collection part, the collections which the rank method is visible to is shown. The translations part shows the various translations in the languages available in Invenio. On the bottom the configuration file is shown, if accessible.</p>
<a name="mr"></a><h3>3.4 Modify rank method</h3>
<p>This interface gives access to modify the bibrank code given when creating the rank method and the configuration file of the rank method, if the file can be accessed. If not, it may not exist, or the httpd user doesn't have enough rights to read the file. On the bottom of the interface, it is possible to choose a template, see it, and copy it over the old rank method configuration if wanted. Remember that the values present in the template is an example, and must be changed where necessary. See this documentation for information about this, and the 'BibRank Internals' link below for additional information.</p>
<a name="dr"></a><h3>3.5 Delete rank method</h3>
<p>If it is necessary to delete a rank method, some precautions must be taken since the configuration of the method will be lost. When deleting a rank method, the configuration file will also be deleted ('invenio/etc/bibrank/bibrankcode.cfg' where bibrankcode is the code of the rank method) if accessible to the httpd user. If not, the file can be deleted manually from console. Any bibrank tasks scheduled to run the deleted rank method must be modified or deleted manually.</p>
<a name="mt"></a><h3>3.6 Modify translations</h3>
<p>If you want to use internalisation of the rank method names, you have to add them using the 'Modify translations' interface. Below a list of all the languages used in the Invenio installation will be shown with the possibility to add the translation for each language.</p>
<a name="mc"></a><h3>3.7 Modify visibility toward collections</h3>
<p>If a rank method should be visible to the users of the Invenio search interface, it must be enabled for one or several collections. A rank method can be visible in the search interface of the whole site, or just one collection. The collections in the upper list box does not show the rank method in the search interface to the user. To change this select the wanted collection and press 'Enable' to enable the rank method for this collection. The collections that the method has been activated for, is shown in the lower list box. To remove a collection, select it and press the 'Disable' button to remove it from the list of collections which the rank method is enabled for.</p>
<a name="bd"></a><h2>4. BibRank Daemon</h2>
<p>The bibrank daemon read the necessary metadata from the Invenio database and combines the read metadata
in different ways to create the ranking data necessary at searchtime to fast be able to rank the results.</p>
<a name="cli1"></a><h3>4.1 Command Line Interface</h3>
<blockquote>
<pre>
Usage bibrank:
bibrank -wjif -a --id=0-30000,30001-860000 --verbose=9
bibrank -wjif -d --modified='2002-10-27 13:57:26'
bibrank -wwrd --recalculate --collection=Articles
bibrank -wwrd -a -i 234-250,293,300-500 -u admin@localhost
Ranking options:
-w, --run=r1[,r2] runs each rank method in the order given
-c, --collection=c1[,c2] select according to collection
-i, --id=low[-high] select according to doc recID
-m, --modified=from[,to] select according to modification date
-l, --lastupdate select according to last update
-a, --add add or update words for selected records
-d, --del delete words for selected records
-S, --stat show statistics for a method
-R, --recalculate recalculate weigth data, used by word frequency method
should be used if ca 1% of the document has been changed
since last time -R was used
Repairing options:
-k, --check check consistency for all records in the table(s)
check if update of ranking data is necessary
-r, --repair try to repair all records in the table(s)
Scheduling options:
-u, --user=USER user name to store task, password needed
-s, --sleeptime=SLEEP time after which to repeat tasks (no)
e.g.: 1s, 30m, 24h, 7d
-t, --time=TIME moment for the task to be active (now)
e.g.: +15s, 5m, 3h , 2002-10-27 13:57:26
General options:
-h, --help print this help and exit
-V, --version print version and exit
-v, --verbose=LEVEL verbose level (from 0 to 9, default 1)
</pre>
</blockquote>
<a name="ubd"></a><h3>4.2 Using BibRank</h3>
<h4>Step 1 - Adding the rank option to the search interface</h4>
<p>To be able to add the needed ranking data to the database, you first have to add the rank method to the database, and
add the wished code you want to use together with it. The name of the configuration file in the next section, needs to
have the same name as the code stored in the database.</p>
<h4>Step 2 - Get necessary external data (ex. jif values)</h4>
<p>Find out what is necessary of data for each method. The bibrankgkb documentation below may be of assistance.</p>
<p><b>Example of necessary data</b> (<code>jif.kb</code> - journal impact factor knowledge base)</p>
<blockquote>
<pre>
Phys. Rev., D---3.838
Phys. Rev. Lett.---6.462
Phys. Lett., B---4.213
Nucl. Instrum. Methods Phys. Res., A---0.964
J. High Energy Phys.---8.664
</pre>
</blockquote>
<h4>Step 3 - Modify the configuration file</h4>
<p>The configuration files for the different rank methods has different option, so verify that you are using the correct
configuration file and rank method. A template for each rank method exists as examples, but may not work on all configurations of Invenio.
For a description of each rank method and the configuration necessary, check section 6 below.</p>
<h4>Step 4 - Add the ranking method as a scheduled task</h4>
<p>When the configuration is okay, you can add the bibrank daemon to the task scheduler using the scheduling options. The daemon can then do a update of the rank method once each day or similar automatically.</p>
<b>Example</b>
<blockquote>
<pre>
$ bibrank -wjif -r
Task #53 was successfully scheduled for execution.
</pre>
</blockquote>
<p>It is adviced to run the BibRank daemon using no parameters, since the default settings then will be used.</p>
<b>Example</b>
<blockquote>
<pre>
$ bibrank
Task #2505 was successfully scheduled for execution.
</pre>
</blockquote>
<h4>Step 5 - Running bibrank manually</h4>
<p>If BibRank is scheduled without any parameters, and no records has been modified, you may get a output like shown below.</p>
<b>Example</b>
<blockquote>
<pre>
$ bibrank 2505
2004-09-07 17:51:46 --> Task #2505 started.
2004-09-07 17:51:46 -->
2004-09-07 17:51:46 --> Running rank method: Number of downloads.
2004-09-07 17:51:47 --> No new records added since last time method was run
2004-09-07 17:52:10 -->
2004-09-07 17:52:10 --> Running rank method: Journal Impact Factor.
2004-09-07 17:52:10 --> No new records added since last time method was run
2004-09-07 17:52:11 --> Reading knowledgebase file: /home/invenio/etc/bibrank/cern_jif.kb
2004-09-07 17:52:11 --> Number of lines read from knowledgebase file: 420
2004-09-07 17:52:11 --> Number of records available in rank method: 0
2004-09-07 17:52:12 -->
2004-09-07 17:52:12 --> Running rank method: Word frequency
2004-09-07 17:52:13 --> rnkWORD01F contains 256842 words from 677912 records
2004-09-07 17:52:14 --> rnkWORD01F is in consistent state
2004-09-07 17:52:14 --> Using the last update time for the rank method
2004-09-07 17:52:14 --> No new records added. rnkWORD01F is up to date
2004-09-07 17:52:14 --> rnkWORD01F contains 256842 words from 677912 records
2004-09-07 17:52:14 --> rnkWORD01F is in consistent state
2004-09-07 17:52:14 --> Task #2505 finished.
</pre>
</blockquote>
<h4>Step 6 - Fast update of modified records</h4>
<p>If you just want to update the latest additions or modified records, you may want to do a faster update by running the daemon without the recalculate option. (the recalculate option is off by default). This may cause lower accurancy when ranking.</p>
<a name="brm"></a><h2>5. BibRank Methods</h2>
<p>Each BibRank method has a configuration file which contains different parameters and sections necessary to do the ranking.</p>
<a name="str"></a><h3>5.1 Single tag rank method</h3>
<p>This method uses one MARC tag together with a file containing possible values for this MARC tag together with a ranking value. This data is used to create a structure containing the record id associated with the ranking value based on the content of the tag. The method can be used for various ways of ranking like ranking by Journal Impact Factor, or use it to let certain authors always appear top of a search.
The parameters needed to be configured for this method is the 'tag','kb_src' and 'check_mandatory_tags'.</p>
<br/><br/><b>Example</b>
<blockquote>
<pre>
[rank_method]
function = single_tag_rank_method
[single_tag_rank]
tag = 909C4p
kb_src = /home/invenio/etc/bibrank/jif.kb
check_mandatory_tags = 909C4c,909C4v,909C4y
</pre>
</blockquote>
<b>Explanation:</b>
<blockquote>
<pre>
[rank_method]
&nbsp; ##The function which is responsible for doing the work. Should not be changed
&nbsp; function = single_tag_rank_method
&nbsp;
&nbsp; ##This section must be available if the single_tag_rank_method is going to be used
&nbsp; [single_tag_kb]
&nbsp;
&nbsp; ##The tag which got the value to be searched for on the left side in the kb file (like the journal name)
&nbsp; tag = 909C4p
&nbsp;
&nbsp; ##The path to the kb file which got the content of the tag above on left side, and value on the left side
&nbsp; kb_src = /home/invenio/etc/bibrank/jif.kb
&nbsp;
&nbsp; ##Tags that must be included for a record to be added to the ranking data, to disable remove tags
&nbsp; check_mandatory_tags = 909C4c,909C4v,909C4y
&nbsp;
</pre>
</blockquote>
<p>The kb_src file must contain data on the form:</p>
<blockquote>
<pre>
Phys. Rev., D---3.838
Phys. Rev. Lett.---6.462
Phys. Lett., B---4.213
Nucl. Instrum. Methods Phys. Res., A---0.964
J. High Energy Phys.---8.664
</pre>
</blockquote>
<p>The left side must match the content of the tag mentioned in the tag variable.
<a name="wrd"></a><h3>5.2 Word Similarity/Similar Records</h3>
<p>The Word Similarity/Similar Records method uses the content of the tags selected to determine which records is most relevant to a query, or most similar to a selected record. This method got a lot of parameters to configure, and it may need some tweaking to get the best result. The BibRank code for this method has to be 'wrd' for it to work. For best result, it is adviced to install the stemming module mentioned in INSTALL, and use a stopword list containing stopwords in the languages the records exists in. The stemmer and stopword list is used to get better results and to limit the size of the index, thus making ranking faster and more accurate. For best result with the stemmer, it is important to mark each tag to be used with the most common language the value of the tag may be in. It is adviced to not change the 'function','table' and the parameters under [find_similar]. If the stemmer is not installed, to assure that no problems exists, the 'stem_if_avail' parameter should be set to 'no'. Each tag to be used by the method has to be given a point. The number of points describes how important one word is in this tag.</p>
<p>When running BibRank to update the index for this rank method, it is not necessary to recalculate each time, but when large number of records has been updated/added, it can be wise to recalculate using the recalculate parameter of BibRank.</p>
<b>Example</b>
<blockquote>
<pre>
[rank_method]
function = word_similarity
[word_similarity]
stemming = en
table = rnkWORD01F
stopword = True
relevance_number_output_prologue = (
relevance_number_output_epilogue = )
&nbsp;#MARC tag,tag points, tag language
tag1 = 6531_a, 2, en
tag2 = 695__a, 1, en
tag3 = 6532_a, 1, en
tag4 = 245__%, 10, en
tag5 = 246_%, 1, fr
tag6 = 250__a, 1, en
tag7 = 711__a, 1, en
tag8 = 210__a, 1, en
tag9 = 222__a, 1, en
tag10 = 520__%, 1, en
tag11 = 590__%, 1, fr
tag12 = 111__a, 1, en
tag13 = 100__%, 2, none
tag14 = 700__%, 1, none
tag15 = 721__a, 1, none
[find_similar]
max_word_occurence = 0.05
min_word_occurence = 0.00
min_word_length = 3
min_nr_words_docs = 3
max_nr_words_upper = 20
max_nr_words_lower = 10
default_min_relevance = 75
</pre>
</blockquote>
<b>Explanation:</b>
<blockquote>
<pre>
[rank_method]
&nbsp;#internal name for the bibrank program, do not modify
function = word_similarity
[word_similarity]
&nbsp;#if stemmer is available, default stemminglanguage should be given here. Adviced to turn off if not installed
stemming = en
&nbsp;#the internal table to load the index tables from.
table = rnkWORD01F
&nbsp;#remove stopwords?
stopword = True
&nbsp;#text to show before the rank value when the search result is presented. &lt;-- to hide result
relevance_number_output_prologue = (
&nbsp;#text to show after the rank value when the search result is presented. --> to hide result
relevance_number_output_epilogue = )
&nbsp;#MARC tag,tag points, tag language
&nbsp;#a list of the tags to be used, together with a number describing the importance of the tag, and the
&nbsp;#most common language for the content. Not all languages are supported. Among the supported ones are:
&nbsp;#fr/french, en/english, no/norwegian, se/swedish, de/german, it/italian, pt/portugese
&nbsp;#keyword
tag1 = 6531_a, 1, en #keyword
tag2 = 695__a, 1, en #keyword
tag3 = 6532_a, 1, en #keyword
tag4 = 245__%, 10, en #title, the words in the title is usually describing a record very good.
tag5 = 246_% , 1, fr #french title
tag6 = 250__a, 1, en #title
tag7 = 711__a, 1, en #title
tag8 = 210__a, 1, en #abbreviated
tag9 = 222__a, 1, en #key title
[find_similar]
&nbsp;#term should exist in maximum X/100% of documents
max_word_occurence = 0.05
&nbsp;#term should exist in minimum X/100% of documents
min_word_occurence = 0.00
&nbsp;#term should be atleast 3 characters long
min_word_length = 3
&nbsp;#term should be in atleast 3 documents or more
min_nr_words_docs = 3
&nbsp;#do not use more than 20 terms for "find similar"
max_nr_words_upper = 20
&nbsp;#if a document contains less than 10 terms, use much used terms too, if not ignore them
max_nr_words_lower = 10
&nbsp;#default minimum relevance value to use for find similar
default_min_relevance = 75
</pre>
</blockquote>
<p>Tip: When executing a search using a ranking method, you can add "verbose=1" to the list of parameteres
in the URL to see which terms have been used in the ranking.</p>
<a name="wrdsolr"></a><h4>5.2.1 Solr Word Similarity/Similar Records</h4>
<p>The Solr Word Similarity/Similar Records method uses Solr to serve word similarity ranking and similar records queries. To use it, the following steps are necessary:</p>
<p>First, Solr is installed:</p>
<blockquote>
<pre>
$ cd &lt;invenio source tree&gt;
$ sudo make install-solrutils
</pre>
</blockquote>
<p>Second, <code>invenio-local.conf</code> is amended:</p>
<blockquote>
<pre>
CFG_SOLR_URL = http://localhost:8983/solr
</pre>
</blockquote>
<p>Third, <code>idxINDEX</code> is amended:</p>
<blockquote>
<pre>
UPDATE idxINDEX SET indexer='SOLR' WHERE name='fulltext'
</pre>
</blockquote>
<p>Fourth, the Solr word similarity ranking method is used:</p>
<blockquote>
<pre>
&lt;invenio installation&gt;/etc/bibrank$ sudo -u www-data cp template_word_similarity_solr.cfg wrd.cfg
</pre>
</blockquote>
<p>Fifth, Solr is started:</p>
<blockquote>
<pre>
&lt;invenio installation&gt;/lib/apache-solr-3.1.0/example$ sudo -u www-data java -jar start.jar
</pre>
</blockquote>
<a name="tdcc"></a><h3>5.3 Time-dependent citation counts</h3>
<p>This ranking method is an extension of the well known "rank by number of citation" method. The difference is that the citations are weighted differently,
based on their publication year. In this way, we can weight more the newly acquired citations, rather than treating them in the same way as older ones.
By doing this, rather than just counting the citations, we can identify highly cited publications that are currently of interest for the scientific community.
</p>
<p>
The different weighting is controlled by the time-decay factor. This factor can have values between 0 and 1. With a time-decay factor of 0 the algorithm will
behave as the classical "rank by number of citations". With a time-decay factor of 1, the algorithm will take into consideration only the citations that come
from documents published in the current year. The time-decay parameter can be adjusted in the configuration file. In order for the algorithm to run, the
appropriate tags for the publication year and the creation date of a document, need to be set.</p>
<p>Please take a look in the configuration file for further explanations.</p>
<a name="lbr"></a><h3>5.4 Link-based ranking</h3>
<p>This ranking method is an extension of the well known pagerank method. Unlike ranking by number of citations, where all the citations are weighed equally,
the link-based ranking weights each citation based on its importance. A high rank for a publication means not necessary that it has been cited a lot, but that
it has been cited by other high ranked publications. In this way it can identify a large number of modestly cited publications that contain important results
for the scientific community. In other words, it associates each publication with an "all-time achievement" rank.
</p>
<p>
In the case of an incomplete citation graph (a lot of citations missing from the repository), the link-based ranking can cause "artificial inflation" of some
of the weights, thus creating errors in ranking. In order to correct this, we advise the use of the external citations (by setting the "use_external_citations"
parameter to "yes"). This will assure the correct propagation of the weight through the network. We use the term of "external links" to denote all the citations
that are missing from the database, and "internal links" to denote all the citations available in the database. The algorithm can be adjusted by changing the
values of the two main parameters (ext_alpha and ext_beta).</p>
<p>For more details, please consult the configuration file.</p>
<a name="tdlbr"></a><h3>5.5 Time-dependent link-based ranking</h3>
<p>This method combines the previous two methods. Its purpose is to highlight important publications that are currently of interest for the scientific community.
This method is not really suited to repositories that allow cycles in their citation graph. Even the bibliographic data sets can allow cycles due to certain
inconsistencies in the publication dates or in the listing of references. Since some of the publications are not dated, the identification/removal of the cycles
can produce a high computational overhead. Because of this and of the link-based ranking which iteratively propagates the weight in the graph, when a strong time
decay factor is used, the newly published documents that are part of a cycle accumulate artificial weight, resulting in an inexact ranking.</p>
<p>For more details, please consult the configuration file.</p>
<a name="cmb"></a><h3>5.6 Combine method</h3>
<p>The 'Combine method' is running each method mentioned in the config file and adding the score together
based on the importance of the method given by the percentage.</p>
<b>Example</b>
<blockquote>
<pre>
[rank_method]
function = combine_method
[combine_method]
method1 = cern_jif,33
method2 = cern_acc,33
method3 = wrd,33
relevance_number_output_prologue = (
relevance_number_output_epilogue = )
</pre>
</blockquote>
<b>Explanation:</b>
<blockquote>
<pre>
[rank_method]
&nbsp;#tells which method to use, do not change
function = combine_method
[combine_method]
&nbsp;#each line tells which method to use, the code is the same as in the BibRank interface, the number describes how
&nbsp;#much of the total score the method should count.
method1 = jif,50
method2 = wrd,50
&nbsp;#text to be shown before the rank value on the search result screen.
relevance_number_output_prologue = (
&nbsp;#text to be shown after the rank value on the search result screen.
relevance_number_output_epilogue = )
</pre>
</blockquote>
<a name="bt"></a><h2>6. bibrankgkb Tool</h2>
<p>For some ranking methods, like the single_tag_rank method, a knowledge base file (kb) with the needed data in the correct format is necessary. This file can be created using the bibrankgkb tool which can read the data either from
the Invenio database, from several web pages using regular expressions, or from another file. In case one source
has another naming convention, bibrank can convert between them using a convert file.</p>
<a name="cli2"></a><h3>6.1 Command Line Interface</h3>
<blockquote>
<pre>
Usage: bibrankgkb %s [options]
Examples:
bibrankgkb --input=bibrankgkb.cfg --output=test.kb
bibrankgkb -otest.cfg -v9
bibrankgkb
Generate options:
-i, --input=file input file, default from /etc/bibrank/bibrankgkb.cfg
-o, --output=file output file, will be placed in current folder
General options:
-h, --help print this help and exit
-V, --version print version and exit
-v, --verbose=LEVEL verbose level (from 0 to 9, default 1)
</pre>
</blockquote>
<a name="ubt"></a><h3>6.2 Using bibrankgkb</h3>
<h4>Step 1 - Find sources</h4>
<p>Since some of the data used for ranking purposes is not freely available, it cannot be bundled with Invenio. To get hold of the necessary data,
you may find it useful to ask your library if they have a copy of the data that can be used (like the Journal Impact Factors from the Science Citation Index), or use google to search the web for any public source.</p>
<h4>Step 2 - Create configuration file</h4>
<p>The default configuration file is shown below.</p>
<blockquote>
<pre>
&nbsp;##The main section
[bibrankgkb]
&nbsp;##The url to a web page with the data to be read, does not need to have the same name as this one, but if there
are several links, the url parameter should end with _0->
url_0 = http://www.taelinke.land.ru/impact_A.html
url_1 = http://www.taelinke.land.ru/impact_B.html
url_2 = http://www.taelinke.land.ru/impact_C.html
url_3 = http://www.taelinke.land.ru/impact_DE.html
url_4 = http://www.taelinke.land.ru/impact_FH.html
url_5 = http://www.taelinke.land.ru/impact_I.html
url_6 = http://www.taelinke.land.ru/impact_J.html
url_7 = http://www.taelinke.land.ru/impact_KN.html
url_8 = http://www.taelinke.land.ru/impact_QQ.html
url_9 = http://www.taelinke.land.ru/impact_RZ.html
&nbsp;##The regular expression for the url mentioned should be given here
url_regexp =
&nbsp;##The various sources that can be read in, can either be a file, web page or from the database
kb_1 = /home/invenio/modules/bibrank/etc/cern_jif.kb
kb_2 = /home/invenio/modules/bibrank/etc/demo_jif.kb
kb_2_filter = /home/invenio/modules/bibrank/etc/convert.kb
kb_3 = SELECT id_bibrec,value FROM bib93x,bibrec_bib93x WHERE tag='938__f' AND id_bibxxx=id
kb_4 = SELECT id_bibrec,value FROM bib21x,bibrec_bib21x WHERE tag='210__a' AND id_bibxxx=id
&nbsp;##This points to the url above (the common part of the url is 'url_' followed by a number
kb_5 = url_%s
&nbsp;##This is the part that will be read by the bibrankgkb tool to determine what to read.
&nbsp;##The first two part (separated by ,,) gives where to look for the conversion file (which convert
&nbsp;##the names between to formats), and the second part is the data source. A conversion file is not
&nbsp;##needed, as shown in create_0. If the source is from a file, url or the database, it must be
&nbsp;##given with file,www or db. If several create lines exists, each will be read in turn, and added
&nbsp;##to a common kb file.
&nbsp;##So this means that:
&nbsp;##create_0: Load from file in variable kb_1 without converting
&nbsp;##create_1: Load from file in variable kb_2 using conversion from file kb_2_filter
&nbsp;##create_3: Load from www using url in variable kb_5 and regular expression in url_regexp
&nbsp;##create_4: Load from database using sql statements in kb_4 and kb_5
create_0 = ,, ,,file,,%(kb_1)s
create_1 = file,,%(kb_2_filter)s,,file,,%(kb_2)s
&nbsp;#create_2 = ,, ,,www,,%(kb_5)s,,%(url_regexp)s
&nbsp;#create_3 = ,, ,,db,,%(kb_4)s,,%(kb_4)s
</pre>
</blockquote>
<p>When you have found a source for the data, created the configuration file, it may be necessary to
create an conversion file, but this depends on the conversions used in the available data versus
the conversion used in your Invenio installation.</p>
<p>The available data may look like this:</p>
<blockquote>
<pre>
COLLOID SURFACE A---1.98
</pre>
</blockquote>
<p>But in Invenio you are using:</p>
<blockquote>
<pre>
Colloids Surf., A---1.98
</pre>
</blockquote>
<p>By using a conversion file like:</p>
<blockquote>
<pre>
COLLOID SURFACE A---Colloids Surf., A
</pre>
</blockquote>
<p>You can convert the source to the correct naming convention.</p>
<blockquote>
<pre>
Colloids Surf., A---1.98
</pre>
</blockquote>
<h4>Step 3 - Run tool</h4>
<p>When ready to run the tool, you may either use the default file (/etc/bibrank/bibrankgkb.cfg), or use another one by giving it using the input variable '--input'.
If you want to test the configuration, you can use '--verbose=9' to output on screen, or if you want to save it to a file, use
'--output=filename', but remember that the file will be saved in the program directory.</p>
<p>The output may look like this:</p>
<blockquote>
<pre>
$ ./bibrankgkb -v9
2004-03-11 17:30:17 --> Running: Generate Knowledge base.
2004-03-11 17:30:17 --> Reading data from file: /home/invenio/etc/bibrank/jif.kb
2004-03-11 17:30:17 --> Reading data from file: /home/invenio/etc/bibrank/conv.kb
2004-03-11 17:30:17 --> Using last resource for converting values.
2004-03-11 17:30:17 --> Reading data from file: /home/invenio/etc/bibrank/jif2.kb
2004-03-11 17:30:17 --> Converting between naming conventions given.
2004-03-11 17:30:17 --> Colloids Surf., A---1.98
2004-03-11 17:30:17 --> Phys. Rev. Lett.---6.462
2004-03-11 17:30:17 --> J. High Energy Phys.---8.664
2004-03-11 17:30:17 --> Nucl. Instrum. Methods Phys. Res., A---0.964
2004-03-11 17:30:17 --> Phys. Lett., B---4.213
2004-03-11 17:30:17 --> Phys. Rev., D---3.838
2004-03-11 17:30:17 --> Total nr of lines: 6
2004-03-11 17:30:17 --> Time used: 0 second(s).
</blockquote>
</pre>
<a name="ainf"></a><h2>7. Additional Information</h2>
<a href="<CFG_SITE_URL>/help/hacking/bibrank-internals">BibRank Internals</a>
c4science · Help