Using the dataset of the German ID's it's tested how good the performance increases, when using only false positive examples during fine tuning. For this all extracted answers have been replaced with the word "<unk>". Thus the network can only learn the structure of the ID-content, but is not teached to extract the correct answer.
With the first fine tuning the performance is increasing (see figure \ref{DE_unks_f1} for the F1 score and figure \ref{DE_unks_tp} for the amount of true positive cases. For some categories such as dates it is sufficient to see the format change in the ID content. They have almost no mistakes after the first fine tuning (???). Other labels, i.e. the "first name" have significant performance increase, but don't reach 100\%. This could be, because the German names follow slightly different "grammar" rules. I.e. in contrast to Switzerland there will be much less influence of French and Italian names and grammar.
For completely new categories, such as the eye color (figure \ref{DE_unk_tp_eye_color}), there is some performance increase, but compared to figure \ref{DE_tp_eye_color} where at least some true positive cases can be used, the amount of true positive cases stays below 10\% of all cases, compared to 50\% after the second fine tuning when using the models extracted answers.
% \caption{After unsupervised fine tuning using 300 ID-card-examples (3000 queries)}
% \label{FR3_f1}
% \end{subfigure}%
\caption{F1 score on German ID-cards. In the sub-figure \ref{CH_unk_f1} the checkpoint trained on the CH-dataset is immediately applied. During fine tuning all extracted answers are replaced the word "<unk>", thus its as if it is a false positive solutions. Nevertheless some performance gain can be observed as i.e. the date format can also be extracted from the ID-content.}
% \caption{After unsupervised fine tuning using 300 ID-card-examples (3000 queries)}
% \label{FR3_tp}
% \end{subfigure}%
\caption{Number of true positive, true negative, false positive and false positive in German ID-cards. In the sub-figure \ref{DE_unk_tp} the checkpoint trained on the CH-dataset is immediately applied. During fine tuning all extracted answers are replaced the word "<unk>", thus its as if it is a false positive solutions. Nevertheless some performance gain can be observed as i.e. the date format can also be extracted from the ID-content.}
% \caption{After unsupervised fine tuning using 300 ID-card-examples (3000 queries)}
% \label{FR3_tp}
% \end{subfigure}%
\caption{Number of true positive, true negative, false positive and false positive in German ID-cards for the label "first name". In the sub-figure \ref{DE_unk_tp} the checkpoint trained on the CH-dataset is immediately applied. During fine tuning all extracted answers are replaced the word "<unk>", thus its as if it is a false positive solutions. Nevertheless some performance gain can be observed as i.e. the date format can also be extracted from the ID-content.}
% \caption{After unsupervised fine tuning using 300 ID-card-examples (3000 queries)}
% \label{FR3_tp}
% \end{subfigure}%
\caption{Number of true positive, true negative, false positive and false positive in German ID-cards for the unknonw label "eye color". In the sub-figure \ref{DE_unk_tp} the checkpoint trained on the CH-dataset is immediately applied. During fine tuning all extracted answers are replaced the word "<unk>", thus its as if it is a false positive solutions. Nevertheless some performance gain can be observed as i.e. the date format can also be extracted from the ID-content.}