Detecting Environmental, Social and Governance (ESG) Topics Using Domain-Specific Language Models and Data Augmentation

Abstract

Despite recent advances in deep learning-based language modelling, many natural language processing (NLP) tasks in the financial domain remain challenging due to the paucity of appropriately labelled data. Other issues that can limit task performance are differences in word distribution between the general corpora – typically used to pre-train language models – and financial corpora, which often exhibit specialized language and symbology. Here, we investigate two approaches that can help to mitigate these issues. Firstly, we experiment with further language model pre-training using large amounts of in-domain data from business and financial news. We then apply augmentation approaches to increase the size of our data-set for model fine-tuning. We report our findings on an Environmental, Social and Governance (ESG) controversies data-set and demonstrate that both approaches are beneficial to accuracy in classification tasks. more

Mehr zum Titel

Titel	Detecting Environmental, Social and Governance (ESG) Topics Using Domain-Specific Language Models and Data Augmentation
Medien	Proceedings of the 14th International Conference on Flexible Query Answering Systems (FQAS 2021), Bratislava, Slovakia, September 19–24, 2021
Verlag	Springer Nature
ISBN	978-3-030-86966-3
Verfasser	Tim Nugent, Nicole Stelea, Prof. Dr. Jochen L. Leidner
Seiten	157-169
Veröffentlichungsdatum	2021-09-16
Zitation	Nugent, Tim; Stelea, Nicole; Leidner, Jochen L. (2021): Detecting Environmental, Social and Governance (ESG) Topics Using Domain-Specific Language Models and Data Augmentation. Proceedings of the 14th International Conference on Flexible Query Answering Systems (FQAS 2021), Bratislava, Slovakia, September 19–24, 2021, 157-169. DOI: 10.1007/978-3-030-86967-0_12