Diffusion warcbase (master) History

R1473/src/main/scala/org/warcbase/spark/rdd/RecordRDD.scala
masterfc0d11495cf135c6c552a2f…

Mar 29 2016

renaming ExtractTopLevelDomain to ExtractDomain
fc0d11495cf1
·ianmilligan1 on Mar 29 2016, 03:11.

Feb 16 2016

Added discardUrlPatterns
8a4c55019413
·Jeremy Wiebe on Feb 16 2016, 16:31.
Added keepUrlPatterns
bf682d8e8efc
·Jeremy Wiebe on Feb 16 2016, 03:54.

Feb 13 2016

Merge branch 'format' for ExtractDate (#154) and TupleFormatter
a7b0e0b07682
·Jeremy Wiebe on Feb 13 2016, 23:01.

Feb 4 2016

Added keepLanguages RDD filter
30a5e5d8d585
·Jeremy Wiebe on Feb 4 2016, 16:36.

Dec 25 2015

add method to filter date by component
4042cac0ad8b
·Alice-Z on Dec 25 2015, 17:49.

Dec 10 2015

Refactor Record API (#189)
2adce498927d
·Alice-Z on Dec 10 2015, 21:59.

Nov 25 2015

Slapped Apache License boilerplate -- now we're a *real* open-source project :)
2e88c1b19afb
·lintool on Nov 25 2015, 03:19.

Nov 21 2015

Clean up, fix tests changed by new keepValidPages
14e521794754
·Alice-Z on Nov 21 2015, 16:33.

Nov 19 2015

Make keepValidPages smarter as in issue #163
2fd98dbfdb67
·Alice-Z on Nov 19 2015, 23:49.

Nov 11 2015

add documentation
a599db1f45b9
·Alice-Z on Nov 11 2015, 22:27.
remove extract methods
1ee0455efdf6
·Alice-Z on Nov 11 2015, 22:24.
Add keepValidPages transformation and layer for counting in Spark
2c2607867ef1
·Alice-Z on Nov 11 2015, 22:08.

Nov 10 2015

Clean up extracting code, use pattern matching
e1be481cd782
·Alice-Z on Nov 10 2015, 02:58.

Nov 9 2015

Clean up enum to function mapping
3957b2cce7fb
·Alice-Z on Nov 9 2015, 18:07.

Nov 3 2015

Add Spark support
8057c46945d0
·Alice-Z on Nov 3 2015, 02:58.