AI-DRIVEN DATA INTEROPERABILITY USING ZERO-SHOT LEARNERS

In 2017, a new network architecture called the Transformer was introduced that leverages a self-attention mechanism to achieve state-of-the-art results in solving sequence-to-sequence tasks in the field of Natural Language Processing (NLP). In this thesis, three deep learning language models using the new transformer architecture (GPT2, RoBERTa and XLM-R) have been fine-tuned to extract information, such as name and birth date from ID-cards. The models are trained on Swiss ID-cards, and are shown to perform well in out-of-sample data such as German or even Finnish ID-cards. The models are able to adapt to unseen datasets by unsupervised fine-tuning. The unseen datasets do not only contain different label names and value representations, but can also contain novel labels with no equivalent during training. The best model extracts information with an average accuracy of 97% in out-of-sample datasets using unsupervised fine-tuning with 200 examples. The models have a zero shot accuracy of roughly 50%. In addition the model is resistant to spelling mistakes and fine-tuning to new databases does not result in catastrophic forgetting.