Homec4science
Diffusion warcbase (master) Graph

R1473/src
master

History Graph

CommitAuthorDetailsCommitted
65b6dc5b7138lintool
Refactor to confirm to /*/ of Wayback to fetch list of available versions. 
Aug 10 2014
096878f5f932lintool
Switched over to 14 digit dates for URLs to align with Wayback. Further… 
Aug 10 2014
7f9764b1a793lintool
Cleaned up servlet fetch code. 
Aug 10 2014
b685d65eba7blintool
Fixed 14 digit date parsing issue (now uses ArchiveUtils); was an issue with… 
Aug 10 2014
bc6aab1ffd21lintool
Fixed a few minor ingestion issues. 
Aug 10 2014
b20ef84e5df9lintool
Refactoring; removing WARC ingestion for now. 
Aug 10 2014
cfe508a831f0lintool
Tweaks to ingest code. 
Aug 10 2014
180b57fa5dc1lintool
Janky, but seems to work: ingesting and serving up raw ARC records. 
Aug 10 2014
8b5a6be9db61lintool
Quick and dirty switch over to webarchive-commons API; stores raw ARC records. 
Aug 10 2014
65c6e54a948eJeffyrao
add UriMappingBuilder Mapreduce version 
Aug 4 2014
59aa95aab857Jeffyrao
fix the bug of selecting webpage by date ineffective 
Jul 25 2014
7175239e0751Jeffyrao
add Hadoop/HBase input choice for ExtractLinks and ExtractSiteLinks classes 
Jul 22 2014
60a9b96beebeMilad Gholami
Merging with master. 
Jun 26 2014
149ce6cf969fMilad Gholami
Fixing git history. 
Jun 26 2014
3b8484a944a8lintool
More work on the admin interface. 
Jun 18 2014
f3015cd7ba4blintool
issue #50 
Jun 18 2014
10d60bd28c75lintool
Fixed broken merge. 
Jun 17 2014
6c452cbb6b5dlintool
Merge branch 'master' into admin 
Jun 17 2014
781fe4247b31lintool
Fixed issue #48 
Jun 17 2014
5f62c2f6fb9dlintool
Added comment. 
Jun 17 2014
d17dddca8deblintool
Minor refactoring. 
Jun 17 2014
ee7f7749a30alintool
Initial working version of anchor text inversion program: issue #43 
Jun 17 2014
02c26d6d8ea4lintool
Started working on issue #46 cleanup of org.warcbase.data.Util 
Jun 17 2014
c7a7247d5fa2lintool
Appears to have fixed issue #49, starting work on admin tool, issue #45. 
Jun 17 2014
a6be4375e0c7lintool
ExtractLinks using HBase appears to be working. 
Jun 13 2014
21a07efb350elintool
Refactored HDFS extractor; HBase extractor still broken. 
Jun 13 2014
bbc73ab64808lintool
Merge branch 'master' into refactoring 
Jun 12 2014
273e5969e943lintool
Light refactoring, pushed column family filter into scan. 
Jun 12 2014
3283bb8512eflintool
Alternative implementation based on iterating over maps... slightly slower. 
Jun 12 2014
dbfbcb0b3c7elintool
More light refactoring. 
Jun 12 2014
34273fdb935aJeffyrao
add hbase option for ExtractLinks 
Jun 12 2014
cfb89d2d3379Jeffyrao
reformat Jinfeng's code 
Jun 12 2014
b02f33c9b43alintool
Refactoring, code cleanup. 
Jun 11 2014
d4c29085ee12lintool
Fixed issue #39 
Jun 11 2014
c3a4348e1250lintool
Debugged HBase scan parameters so that they don't knock over region servers… 
Jun 11 2014
0e61a3094550milad621
Issue 38 fixed. Still need to add other URL Encoding characters. 
Jun 6 2014
20ef503dbfe5lintool
Moving ExtractLinks and ExtractSiteLinks into analysis.graph package, per Issue… 
Jun 5 2014
dc58365a9579lintool
Extracts values at different timestamps. 
Jun 5 2014
60b827d1c174lintool
Refactored getIdRange method signature, add more test cases to UriMapping. 
Jun 5 2014
a6b17787ccd1lintool
Merge branch 'extract-links' of github.com:Jeffyrao/warcbase into refactoring 
Jun 5 2014
7fc1b92d60a7Jeffyrao
fix issue 40 that UriMapping prefix search should return empty result when no… 
Jun 5 2014
a8953616248clintool
Refactoring, added test case (currently broken). 
Jun 4 2014
6518836b7ea0Jeffyrao
fix issue 32, update ExtractSiteLinks code 
Jun 3 2014
a17e3c74954aJeffyrao
fix issue 30, add ExtractSiteLinks code 
May 29 2014
3f36eff4f429Jeffyrao
fix issue 31 
May 27 2014
b4ab2ea0d499lintool
Initial MapReduce over HBase demo. 
May 26 2014
0598dcd91073Jeffyrao
more edits 
May 25 2014
cad5eddceb19Jeffyrao
remove javacsv dependency, add opencsv dependency 
May 25 2014
f0e36b46b872Jinfeng Rao
Merge remote-tracking branch 'upstream/master' into extract-links 
May 25 2014
7c21e2df9267lintool
Fixed robustness and OOM issues when ingesting corrupt ARC files. 
May 23 2014
67888b472702milad621
Fixed issue29. 
May 23 2014
fcc4acf98a6flintool
Simple program to find URL patterns in archives. 
May 22 2014
52b39cfecc82lintool
Minor refactoring. 
May 6 2014
0856f3faddd1lintool
Merge branch 'master' of github.com:milad621/warcbase into display_issues 
May 6 2014
a4247934797cmilad621
changed type of nobanner to boolean. 
May 5 2014
c5432ac2468cmilad621
Working on issue17. Added /noframes to the url but still needs to add warcbase… 
May 4 2014
8799707e9856milad621
Fixed issue 25 
Apr 24 2014
24d7d8e08807Jeffyrao
modified ExtractSiteLinks.java, changed to read csv format of prefix input 
Apr 20 2014
627743751578milad621
juniversalchardet removed. 
Apr 19 2014
0d34860367d9milad621
For issue17 it now shows a page with three frames, each has a seperate search… 
Apr 19 2014
8d9ef6d2ac6bJeffyrao
add extract site-level links 
Apr 16 2014
fd511b6eaf7bmilad621
Fixed Issue 24. 
Apr 6 2014
997a8e84e816lintool
Exposed -getPrefix command-line option. 
Mar 31 2014
758948b463b9lintool
Light refactoring, fixed a few errors. 
Mar 31 2014
5227ba65232clintool
Merge branch 'extract-links' of github.com:Jeffyrao/warcbase into fst 
Mar 31 2014
e3ccb37b2e1dJeffyrao
remove TestFSTPrefix class 
Mar 31 2014
71c68ecae0faJeffyrao
add prefix search feature to UriMapping class; add test class for UriMapping… 
Mar 31 2014
2f1574575438lintool
More work on Issue #22: prev/next buttons turns on and off appropriately. 
Mar 29 2014
a48c65e498abJeffyrao
add Lucene FST prefix search 
Mar 28 2014
6d1747bd14afmilad621
Fixed issue 18. 
Mar 28 2014
88e5d97286e4milad621
Fixed issues 21 and 22. 
Mar 28 2014
dac5827fa1e5milad621
Merge branch 'master' of https://github.com/lintool/warcbase 
Mar 28 2014
f2f1e34837f3milad621
merged with master 
Mar 28 2014
3da8131e077blintool
Minor code cleanup. 
Mar 27 2014
d965c015a914milad621
Bug with getting close dates fixed. 
Mar 26 2014
be5a4d366786lintool
Fixed bug with MAX_VERSION blowing up array length. 
Mar 25 2014
72b9558b3ca5lintool
Merge branch 'issue12' of github.com:milad621/warcbase into multi_version_cell 
Mar 25 2014
ef6650cb7bc4milad621
fixing branches 
Mar 25 2014
0d43c98ee177milad621
Fixed issue12 and some issues with dates in browser. 
Mar 25 2014
6c48ce81ee80milad621
Issue12 fixed 
Mar 24 2014
e0d6be1055f4milad621
test push 
Mar 24 2014
252eb87988b1milad621
issue12 fixed 
Mar 24 2014
24370c4f051fJeffyrao
update README file, add steps for getting URLs and extracting links 
Mar 21 2014
cb9e68d4f94dJeffyrao
add ExtractLinks and PrefixQuery(ExtractSiteLinks) 
Mar 20 2014
374b6f2c2173lintool
Adds Snappy compression. 
Mar 19 2014
232a07ac56c9cneud
Merge remote-tracking branch 'lintool/master' 
Mar 18 2014
303860c9ae3blintool
Reformatting source. 
Mar 17 2014
c52474128b38lintool
Merge branch 'disable_wal' of github.com:lintool/warcbase into disable_wal 
Mar 17 2014
d543a4957916lintool
Merge branch 'master' into disable_wal 
Mar 17 2014
b430dc7c94f6lintool
Merge branch 'jar_upgrade' into disable_wal 
Mar 17 2014
8c649e5336aflintool
Fixed Issue #7 
Mar 17 2014
48ac86cd8479lintool
Added log4j properties. 
Mar 17 2014
a33f92f340e1lintool
Added try/catch block in ingestion code. 
Mar 17 2014
53aacb335403Jeffyrao
code reformat via eclipse imported project 
Mar 17 2014
987f9518ebb8lintool
Ignores all files that aren't WARC/ARC files in ingest. 
Mar 16 2014
dfb63a169b43lintool
Tried disabling WAL to see impact on performance. 
Mar 16 2014
1ad985045d9aJeffyrao
code reformat 
Mar 16 2014
073fd3206185lintool
Simplified Maven dependencies; Moved from IA artifacts to openwayback artifacts. 
Mar 16 2014
228ca87cd0c3lintool
Light refactoring. 
Mar 16 2014
19453fc7ac8flintool
whitespace 
Mar 16 2014
Prev
1
2
3
Next
c4science · Help