Diffusion warcbase (master) History

R1473/src/test
master4ceefef78cdc934768cc4c6…

Aug 30 2014

Figured out how to extract MIME type and date from WARC.
4798a4314b6e
·lintool on Aug 30 2014, 03:10.

Aug 29 2014

Added test cases to try loading WARC records from a stream; back-ported same…
f3516c7fd7f0
·lintool on Aug 29 2014, 04:15.

Aug 28 2014

Added simple test case.
28c5c007f4fd
·lintool on Aug 28 2014, 05:48.
WARC sample from https://archive.org/details/ExampleArcAndWarcFiles
8e67d49d44b3
·lintool on Aug 28 2014, 05:19.

Aug 23 2014

Implemented ArcRecordWritable.
ccee8fd2204a
·lintool on Aug 23 2014, 05:25.
Minor refactoring.
846ebb216100
·lintool on Aug 23 2014, 03:46.

Aug 22 2014

Added Hadoop bindings for webarchive-commons ARC readers, demo, test cases.
47f3c46c099d
·lintool on Aug 22 2014, 14:28.
Added/refactored test cases for JWAT.
b07376aba3fe
·lintool on Aug 22 2014, 13:56.

Aug 14 2014

Refactoring UrlUtil and related classes.
bf606c07d453
·lintool on Aug 14 2014, 23:31.

Aug 12 2014

Fixed compile and broken test issues.
93f6d42f4ff4
·lintool on Aug 12 2014, 21:47.
Merge branch 'master' of github.com:cneud/warcbase into cneud-integration
accd1978862d
·lintool on Aug 12 2014, 19:38.

Jun 18 2014

issue #50
f3015cd7ba4b
·lintool on Jun 18 2014, 19:57.

Jun 17 2014

Started working on issue #46 cleanup of org.warcbase.data.Util
02c26d6d8ea4
·lintool on Jun 17 2014, 17:20.

Jun 13 2014

ExtractLinks using HBase appears to be working.
a6be4375e0c7
·lintool on Jun 13 2014, 03:08.

Jun 5 2014

Refactored getIdRange method signature, add more test cases to UriMapping.
60b827d1c174
·lintool on Jun 5 2014, 16:16.
fix issue 40 that UriMapping prefix search should return empty result when no…
7fc1b92d60a7
·Jeffyrao on Jun 5 2014, 03:37.

Jun 4 2014

Refactoring, added test case (currently broken).
a8953616248c
·lintool on Jun 4 2014, 21:12.

Mar 31 2014

Light refactoring, fixed a few errors.
758948b463b9
·lintool on Mar 31 2014, 17:56.
add prefix search feature to UriMapping class; add test class for UriMapping…
71c68ecae0fa
·Jeffyrao on Mar 31 2014, 05:32.

Dec 19 2013

Added a null pointer check and a more Pig friendly return value from the UDFs
db4ebe9f4826
·pmd on Dec 19 2013, 15:48.
Refactored the configuration of the magic lib into the Pig script.
ba201c27e210
·pmd on Dec 19 2013, 15:23.
Improved the DetectMimeTypeTika Pig script.
71b90c81859e
·pmd on Dec 19 2013, 15:14.
Corrected an error in the DetectMimeTypeMagic Pig script and the corresponding…
bedac9288080
·pmd on Dec 19 2013, 15:07.
Removed warnings
5d312a3f80ec
·pmd on Dec 19 2013, 14:29.
Refactored the DetectMimeType into two seperate methods: one for each detection…
b3fdd488fdcb
·pmd on Dec 19 2013, 14:22.

Dec 10 2013

Corrected a comment
842012e0de5f
·pmd on Dec 10 2013, 11:51.
Changed unit test to match the change in ArcLoader that removed the filter for…
e71f20141a36
·pmd on Dec 10 2013, 11:47.
Changed unit test to match the change in ArcLoader that removed the filter for…
10b02ec31780
·pmd on Dec 10 2013, 11:25.

Dec 9 2013

Use the provided ARC file for the unit test.
b8f63da4d685
·pmd on Dec 9 2013, 15:18.
Improving the unit test of the DetectMimeType by using the two identification…
25ca4df65a50
·pmd on Dec 9 2013, 15:17.
Add the magic lib UDF to the Pig script
49f50dd95873
·pmd on Dec 9 2013, 15:13.

Dec 4 2013

First version of a magic lib UDF
1a209cf5b04a
·pmd on Dec 4 2013, 14:43.

Dec 3 2013

Added unit test for the language detection UDF.
c0e996ec2e61
·pmd on Dec 3 2013, 15:03.

Dec 2 2013

Cleaned up Pig test cases, added JWAT test case.
a4282d81d6e9
·lintool on Dec 2 2013, 17:33.
added a pig unit test
17bc9616a180
·graemon on Dec 2 2013, 16:28.
added a pig unit test
8885f5db4ded
·graemon on Dec 2 2013, 14:55.