Diffusion warcbase (master) History

R1473/src/main/scala/org/warcbase/spark/scripts/CrawlStatistics.scala
master71a19fbaaa743eb9e633fc6…

Apr 21 2016

more tweaks for case sensitive getCrawlDate
d66f001374b1
·ianmilligan1 on Apr 21 2016, 17:51.

Mar 29 2016

Fixed compile bug, converted to .removePrefixWWW()
513c69695940
·lintool on Mar 29 2016, 05:10.
renaming ExtractTopLevelDomain to ExtractDomain
fc0d11495cf1
·ianmilligan1 on Mar 29 2016, 03:11.

Feb 13 2016

Use shapeless to flatten tuples of any arity
4b8f0fb7b482
·Jeremy Wiebe on Feb 13 2016, 06:48.

Dec 25 2015

Cleanup and add comments
d1c3f5d49783
·Alice-Z on Dec 25 2015, 17:31.
Add Date utils to clarify date extraction
1ec6e05d7540
·Alice-Z on Dec 25 2015, 17:31.
Add Formatter to un-nest tuples and print in tab-delimited format
c773fd242ff5
·Alice-Z on Dec 25 2015, 17:28.

Dec 10 2015

Use SerializableWritable wrapper for serialization
042253b579cc
·Alice-Z on Dec 10 2015, 22:03.
Refactor Record API (#189)
2adce498927d
·Alice-Z on Dec 10 2015, 21:59.

Dec 3 2015

Fix WARecord getContentString
90616dd7b3bd
·Alice-Z on Dec 3 2015, 17:11.

Dec 1 2015

Add example scripts for crawl statistics [Issue #182]
43b9b1f84ef0
·Alice-Z on Dec 1 2015, 02:20.
Add example scripts for crawl statistics and social media links
ba91869cd370
·Alice-Z on Dec 1 2015, 01:18.