Diffusion cop-mining-participants (master) History

cop-mining-participants/code
master9dd37a46c53e4491cdd7688…

Nov 10 2020

Minor changes and todo
9dd37a46c53e
·Jan Linder on Nov 10 2020, 11:07.
Add experience graph s.t. it works
fa5ec0d659bc
·Jan Linder on Nov 10 2020, 08:49.

Nov 9 2020

Added experience graph but failed
4d6352e6b52f
·Jan Linder on Nov 9 2020, 22:10.
Add file to find average number of participants
90dfb209b07e
·Jan Linder on Nov 9 2020, 21:37.
Add delegation size plot
d3111e1d6608
·Jan Linder on Nov 9 2020, 17:25.
add gender analysis
9484071d42bd
·Jan Linder on Nov 9 2020, 16:13.

Nov 3 2020

Add hasTitle to complete dataset
5414f53caa32
·Jan Linder on Nov 3 2020, 11:34.
Add gender in the complete dataset
50cb623d6541
·Jan Linder on Nov 3 2020, 11:21.
Make other Analyzers unstatic.
ed5b12fba5f8
·Jan Linder on Nov 3 2020, 09:41.
Made Analyzer 1-5 unstatic and solved issue with encoding for cop5
85156b2d7027
·Jan Linder on Nov 3 2020, 08:49.

Nov 2 2020

Extraction of SB now possible. Now you need to give meeting label to…
71ee7f02893d
·Jan Linder on Nov 2 2020, 16:58.
Restructuring of repo
2eecd75d75b8
·Jan Linder on Nov 2 2020, 15:50.

Oct 26 2020

Add the description lines extraction
a46d37c100b6
·Jan Linder on Oct 26 2020, 21:44.
Add translation from french to english for cop2
73d06bab31b6
·Jan Linder on Oct 26 2020, 20:17.
Generate the complete dataset now possible
b5c1185e3a2e
·Jan Linder on Oct 26 2020, 20:17.

Oct 22 2020

Add the possibility to have more than one affiliation category per page
db6864b1a785
·Jan Linder on Oct 22 2020, 22:48.
Added new issues in source to readme
0ad97826fa8a
·Jan Linder on Oct 22 2020, 22:47.
Add presentation for Paula and Marlene, week6
2e7f25ecd1bc
·Jan Linder on Oct 22 2020, 16:11.

Oct 21 2020

add generate plots
5531d854d21a
·Jan Linder on Oct 21 2020, 20:43.

Oct 20 2020

Minor bugfixes and improvements
62f11a926efb
·Jan Linder on Oct 20 2020, 23:20.
Explain Analyzer in readme
0b5560f27517
·Jan Linder on Oct 20 2020, 22:55.
Rename cop1to5, included cop5 there
3fd706cf5bd0
·Jan Linder on Oct 20 2020, 22:50.
Improve Analyzers, especially cop7to8
97b00603c9f4
·Jan Linder on Oct 20 2020, 22:48.

Oct 19 2020

Implement cop7to8_analyzer and affilition list extractor.
67a5ee11403c
·Jan Linder on Oct 19 2020, 18:52.
Added party recognition for all cops
883bce5e3184
·Jan Linder on Oct 19 2020, 17:29.

Oct 17 2020

First edition of affiliation category extraction. Only works for cop5+ yet
8b9912f6d1fa
·Jan Linder on Oct 17 2020, 14:07.

Oct 14 2020

minor adaption op salutory addresses
fa3369344206
·Jan Linder on Oct 14 2020, 09:13.
Make pdfToText unstatic.
2186bf7609de
·Jan Linder on Oct 14 2020, 08:24.

Oct 13 2020

Add todo week5
7514566bb3e2
·Jan Linder on Oct 13 2020, 21:11.

Oct 12 2020

Improve pdftotext and its analysis.
dc91f390862e
·Jan Linder on Oct 12 2020, 22:39.
Implement pdfToTxt manually quite correctly
e22f3e528e6a
·Jan Linder on Oct 12 2020, 12:48.

Oct 10 2020

implement PDFPageDetailedAggregator to get the positions of LTContainers
a0f6ce99d856
·Jan Linder on Oct 10 2020, 12:14.

Oct 6 2020

bring copnewer_analyzer to work
c66cb28f7eac
·Jan Linder on Oct 6 2020, 17:32.
Remove old code files
0f3b244e9c77
·Jan Linder on Oct 6 2020, 13:50.
IMPLEMENT PDF TO TXT CORRECTLY Now use pdfminer with the laparam argument to…
29e001a8ff57
·Jan Linder on Oct 6 2020, 00:36.

Oct 5 2020

the ocr now works correctly
66d3e48b2e2a
·Jan Linder on Oct 5 2020, 21:38.
blacklist
154caa57dcc3
·Jan Linder on Oct 5 2020, 21:12.
whitelist ocr: not perfect
2f69d7e02c12
·Jan Linder on Oct 5 2020, 20:41.
Improve analysis for cop3 and cop4
9cc36860b206
·Jan Linder on Oct 5 2020, 14:16.
Small corrections for OCR
0d36b15c21a8
·Jan Linder on Oct 5 2020, 13:16.
finish modularization
67d367f6243d
·Jan Linder on Oct 5 2020, 00:14.

Oct 4 2020

File structure updated. (untested)
e2ddd912c5b8
·Jan Linder on Oct 4 2020, 15:15.
Begin with making a proper modularization
26f03fd369c3
·Jan Linder on Oct 4 2020, 14:22.

Sep 30 2020

inserted boxes for OCR and right parameters
2d93960b8990
·Jan Linder on Sep 30 2020, 21:52.

Sep 28 2020

minor changes of process, added raw of cop3
d37074c1d20f
·Jan Linder on Sep 28 2020, 16:55.
progress on cop2-4
5b6aa3b3ac8b
·Jan Linder on Sep 28 2020, 15:55.

Sep 27 2020

Use of process_copX.py precised in README
57cdc59e9210
·Jan Linder on Sep 27 2020, 12:16.
Implemented processing of cop2-4. Works good for countries, but has major…
355f0c163915
·Jan Linder on Sep 27 2020, 11:46.
try with pypdf2
a557a870c334
·Jan Linder on Sep 27 2020, 09:47.

Sep 26 2020

implemented process cop for 5 - 25 with textract but there are major errors in…
13691532eec5
·Jan Linder on Sep 26 2020, 18:44.

Sep 23 2020

progress on the class and process script
7f3f311f331c
·Jan Linder on Sep 23 2020, 08:40.

Sep 22 2020

Began the copx file
c747832a97fe
·Jan Linder on Sep 22 2020, 17:28.
added raw txt for cop25
aa92caa65e57
·Jan Linder on Sep 22 2020, 09:44.

Sep 17 2020

data complete
9514a37465f4
·Jan Linder on Sep 17 2020, 15:00.
data complete
77b17d5e7bd6
·Jan Linder on Sep 17 2020, 15:00.
first part of the lists
45310b6c42f1
·Jan Linder on Sep 17 2020, 11:34.