Homec4science
Diffusion warcbase (master) Graph

R1473/src
master

History Graph

CommitAuthorDetailsCommitted
10d60bd28c75lintool
Fixed broken merge. 
Jun 17 2014
6c452cbb6b5dlintool
Merge branch 'master' into admin 
Jun 17 2014
781fe4247b31lintool
Fixed issue #48 
Jun 17 2014
5f62c2f6fb9dlintool
Added comment. 
Jun 17 2014
d17dddca8deblintool
Minor refactoring. 
Jun 17 2014
ee7f7749a30alintool
Initial working version of anchor text inversion program: issue #43 
Jun 17 2014
02c26d6d8ea4lintool
Started working on issue #46 cleanup of org.warcbase.data.Util 
Jun 17 2014
c7a7247d5fa2lintool
Appears to have fixed issue #49, starting work on admin tool, issue #45. 
Jun 17 2014
a6be4375e0c7lintool
ExtractLinks using HBase appears to be working. 
Jun 13 2014
21a07efb350elintool
Refactored HDFS extractor; HBase extractor still broken. 
Jun 13 2014
bbc73ab64808lintool
Merge branch 'master' into refactoring 
Jun 12 2014
273e5969e943lintool
Light refactoring, pushed column family filter into scan. 
Jun 12 2014
3283bb8512eflintool
Alternative implementation based on iterating over maps... slightly slower. 
Jun 12 2014
dbfbcb0b3c7elintool
More light refactoring. 
Jun 12 2014
34273fdb935aJeffyrao
add hbase option for ExtractLinks 
Jun 12 2014
cfb89d2d3379Jeffyrao
reformat Jinfeng's code 
Jun 12 2014
b02f33c9b43alintool
Refactoring, code cleanup. 
Jun 11 2014
d4c29085ee12lintool
Fixed issue #39 
Jun 11 2014
c3a4348e1250lintool
Debugged HBase scan parameters so that they don't knock over region servers… 
Jun 11 2014
0e61a3094550milad621
Issue 38 fixed. Still need to add other URL Encoding characters. 
Jun 6 2014
20ef503dbfe5lintool
Moving ExtractLinks and ExtractSiteLinks into analysis.graph package, per Issue… 
Jun 5 2014
dc58365a9579lintool
Extracts values at different timestamps. 
Jun 5 2014
60b827d1c174lintool
Refactored getIdRange method signature, add more test cases to UriMapping. 
Jun 5 2014
a6b17787ccd1lintool
Merge branch 'extract-links' of github.com:Jeffyrao/warcbase into refactoring 
Jun 5 2014
7fc1b92d60a7Jeffyrao
fix issue 40 that UriMapping prefix search should return empty result when no… 
Jun 5 2014
a8953616248clintool
Refactoring, added test case (currently broken). 
Jun 4 2014
6518836b7ea0Jeffyrao
fix issue 32, update ExtractSiteLinks code 
Jun 3 2014
a17e3c74954aJeffyrao
fix issue 30, add ExtractSiteLinks code 
May 29 2014
3f36eff4f429Jeffyrao
fix issue 31 
May 27 2014
b4ab2ea0d499lintool
Initial MapReduce over HBase demo. 
May 26 2014
0598dcd91073Jeffyrao
more edits 
May 25 2014
cad5eddceb19Jeffyrao
remove javacsv dependency, add opencsv dependency 
May 25 2014
f0e36b46b872Jinfeng Rao
Merge remote-tracking branch 'upstream/master' into extract-links 
May 25 2014
7c21e2df9267lintool
Fixed robustness and OOM issues when ingesting corrupt ARC files. 
May 23 2014
67888b472702milad621
Fixed issue29. 
May 23 2014
fcc4acf98a6flintool
Simple program to find URL patterns in archives. 
May 22 2014
52b39cfecc82lintool
Minor refactoring. 
May 6 2014
0856f3faddd1lintool
Merge branch 'master' of github.com:milad621/warcbase into display_issues 
May 6 2014
a4247934797cmilad621
changed type of nobanner to boolean. 
May 5 2014
c5432ac2468cmilad621
Working on issue17. Added /noframes to the url but still needs to add warcbase… 
May 4 2014
8799707e9856milad621
Fixed issue 25 
Apr 24 2014
24d7d8e08807Jeffyrao
modified ExtractSiteLinks.java, changed to read csv format of prefix input 
Apr 20 2014
627743751578milad621
juniversalchardet removed. 
Apr 19 2014
0d34860367d9milad621
For issue17 it now shows a page with three frames, each has a seperate search… 
Apr 19 2014
8d9ef6d2ac6bJeffyrao
add extract site-level links 
Apr 16 2014
fd511b6eaf7bmilad621
Fixed Issue 24. 
Apr 6 2014
997a8e84e816lintool
Exposed -getPrefix command-line option. 
Mar 31 2014
758948b463b9lintool
Light refactoring, fixed a few errors. 
Mar 31 2014
5227ba65232clintool
Merge branch 'extract-links' of github.com:Jeffyrao/warcbase into fst 
Mar 31 2014
e3ccb37b2e1dJeffyrao
remove TestFSTPrefix class 
Mar 31 2014
71c68ecae0faJeffyrao
add prefix search feature to UriMapping class; add test class for UriMapping… 
Mar 31 2014
2f1574575438lintool
More work on Issue #22: prev/next buttons turns on and off appropriately. 
Mar 29 2014
a48c65e498abJeffyrao
add Lucene FST prefix search 
Mar 28 2014
6d1747bd14afmilad621
Fixed issue 18. 
Mar 28 2014
88e5d97286e4milad621
Fixed issues 21 and 22. 
Mar 28 2014
dac5827fa1e5milad621
Merge branch 'master' of https://github.com/lintool/warcbase 
Mar 28 2014
f2f1e34837f3milad621
merged with master 
Mar 28 2014
3da8131e077blintool
Minor code cleanup. 
Mar 27 2014
d965c015a914milad621
Bug with getting close dates fixed. 
Mar 26 2014
be5a4d366786lintool
Fixed bug with MAX_VERSION blowing up array length. 
Mar 25 2014
72b9558b3ca5lintool
Merge branch 'issue12' of github.com:milad621/warcbase into multi_version_cell 
Mar 25 2014
ef6650cb7bc4milad621
fixing branches 
Mar 25 2014
0d43c98ee177milad621
Fixed issue12 and some issues with dates in browser. 
Mar 25 2014
6c48ce81ee80milad621
Issue12 fixed 
Mar 24 2014
e0d6be1055f4milad621
test push 
Mar 24 2014
252eb87988b1milad621
issue12 fixed 
Mar 24 2014
24370c4f051fJeffyrao
update README file, add steps for getting URLs and extracting links 
Mar 21 2014
cb9e68d4f94dJeffyrao
add ExtractLinks and PrefixQuery(ExtractSiteLinks) 
Mar 20 2014
374b6f2c2173lintool
Adds Snappy compression. 
Mar 19 2014
232a07ac56c9cneud
Merge remote-tracking branch 'lintool/master' 
Mar 18 2014
303860c9ae3blintool
Reformatting source. 
Mar 17 2014
c52474128b38lintool
Merge branch 'disable_wal' of github.com:lintool/warcbase into disable_wal 
Mar 17 2014
d543a4957916lintool
Merge branch 'master' into disable_wal 
Mar 17 2014
b430dc7c94f6lintool
Merge branch 'jar_upgrade' into disable_wal 
Mar 17 2014
8c649e5336aflintool
Fixed Issue #7 
Mar 17 2014
48ac86cd8479lintool
Added log4j properties. 
Mar 17 2014
a33f92f340e1lintool
Added try/catch block in ingestion code. 
Mar 17 2014
53aacb335403Jeffyrao
code reformat via eclipse imported project 
Mar 17 2014
987f9518ebb8lintool
Ignores all files that aren't WARC/ARC files in ingest. 
Mar 16 2014
dfb63a169b43lintool
Tried disabling WAL to see impact on performance. 
Mar 16 2014
1ad985045d9aJeffyrao
code reformat 
Mar 16 2014
073fd3206185lintool
Simplified Maven dependencies; Moved from IA artifacts to openwayback artifacts. 
Mar 16 2014
228ca87cd0c3lintool
Light refactoring. 
Mar 16 2014
19453fc7ac8flintool
whitespace 
Mar 16 2014
1e0783ff8a6clintool
Merge branch 'new_hbase_structure' of https://github.com/milad621/warcbase into… 
Mar 16 2014
fdb75decbdf4lintool
Whitespace. 
Mar 15 2014
d86dcb73c854lintool
Merge branch 'extract-links' of https://github.com/Jeffyrao/warcbase into… 
Mar 15 2014
75a0820c382emilad621
servlet updated with the new hbase structure. 
Dec 28 2013
db4ebe9f4826pmd
Added a null pointer check and a more Pig friendly return value from the UDFs 
Dec 19 2013
ba201c27e210pmd
Refactored the configuration of the magic lib into the Pig script. 
Dec 19 2013
71b90c81859epmd
Improved the DetectMimeTypeTika Pig script. 
Dec 19 2013
bedac9288080pmd
Corrected an error in the DetectMimeTypeMagic Pig script and the corresponding… 
Dec 19 2013
5d312a3f80ecpmd
Removed warnings 
Dec 19 2013
b3fdd488fdcbpmd
Refactored the DetectMimeType into two seperate methods: one for each detection… 
Dec 19 2013
a80e87ca6f74milad621
new hbase structure for ingest files 
Dec 11 2013
d6c0cec7efb4pmd
Added a TODO comment 
Dec 11 2013
842012e0de5fpmd
Corrected a comment 
Dec 10 2013
e71f20141a36pmd
Changed unit test to match the change in ArcLoader that removed the filter for… 
Dec 10 2013
10b02ec31780pmd
Changed unit test to match the change in ArcLoader that removed the filter for… 
Dec 10 2013
b8f63da4d685pmd
Use the provided ARC file for the unit test. 
Dec 9 2013
Prev
1
2
3
Next
c4science · Help