Efficient calculation of sentence semantic similarity: a proposed scheme based on machine learning approaches and NLP techniques

Abstract

Sentence semantic similarity plays a crucial role in a variety of applications such as Machine Translation, Information Retrieval, Question Answering and Multi-document Summarization. Considering the variability of natural language expression, sentence semantic similarity detection is not a trivial task. This paper tries to make use of Natural Language Processing (NLP) as well as machine learning techniques in order to propose a scheme for sentence semantic similarity. In the first part of the proposed scheme, i.e., the NLP section, different sets of linguistic features including string-based, semantic-based, Named Entity-based and syntax-based features are extracted. In the second part, machine learning algorithms are used to construct classification models on the extracted set of features. Experimental results in the first part indicate that extracted features are valid for sentence semantic similarity. Moreover, by comparing the performance of different classification algorithms in the second part, KNN seems to be the most successful algorithm. Overall, experimental results indicate that the proposed approach can be used to improve the performance of sentence semantic similarity detection especially in terms of accuracy.