Fixed issues #96, #98, #99
df7f8711c64f
Actions

Authored by lintool <jimmylin@umd.edu> on Oct 31 2014, 18:32.

Description

Fixed issues #96, #98, #99

Issue #99 Implement WacWarcInputFormat
Issue #98 Implement WarcRecordWritable
Issue #96 Pig loader should export it's own schema

Committed

lintool <jimmylin@umd.edu>

Oct 31 2014, 18:32

Pushed

dportabella

Oct 19 2016, 16:29

Parents

Branches

Unknown

Tags

Unknown

lintool <jimmylin@umd.edu> committed R1473:df7f8711c64f: Fixed issues #96, #98, #99 (authored by lintool <jimmylin@umd.edu>).Oct 31 2014, 18:32

Commit	Author	Details	Committed
b8915343f5c9	lintool	Minor refactoring.	Oct 31 2014
0145d74ed35e	lintool	Refactoring to create method that extracts MIME from WARC response records.	Oct 22 2014
d491020c8287	lintool	Merge branch 'pig' into warc	Oct 19 2014
410cfd81a069	lintool	WARC-related Hadoop bindings.	Oct 19 2014
f4a249469cbc	lintool	Minor refactoring.	Oct 19 2014
0c52caed50cf	lintool	Pig ArcLoader exports its own ResourceSchema.	Oct 19 2014

				Path
	M			src/main/java/org/warcbase/data/WarcRecordUtils.java
	P			src/main/java/org/warcbase/io/WarcRecordWritable.java Copied from src/main/java/org/warcbase/io/ArcRecordWritable.java
	P			src/main/java/org/warcbase/mapreduce/WacArcInputFormat.java Copied to src/main/java/org/warcbase/mapreduce/WacWarcInputFormat.java
	P			src/main/java/org/warcbase/mapreduce/WacWarcInputFormat.java Copied from src/main/java/org/warcbase/mapreduce/WacArcInputFormat.java
	M			src/main/java/org/warcbase/pig/ArcLoader.java
	M			src/test/java/org/warcbase/ingest/WacWarcLoaderTest.java
	P			src/test/java/org/warcbase/io/ArcRecordWritableTest.java Copied to src/test/java/org/warcbase/io/WarcRecordWritableTest.java
	P			src/test/java/org/warcbase/io/WarcRecordWritableTest.java Copied from src/test/java/org/warcbase/io/ArcRecordWritableTest.java
	P			src/test/java/org/warcbase/mapreduce/WacWarcInputFormatTest.java Copied from src/test/java/org/warcbase/mapreduce/JwatArcInputFormatTest.java
	M			src/test/resources/scripts/TestArcLoader.pig
	M			src/test/resources/scripts/TestCountLinks.pig
	M			src/test/resources/scripts/TestDetectLanguage.pig
	M			src/test/resources/scripts/TestDetectMimeTypeTika.pig