The Effect of Stop Word Removal and Stemming I n Datapreprocessing

Abstract

Preprocessing is one the important technique in text mining and its application. It helps in
converting the original textual data to most significant text features. The main objective of preprocessing
is to split the sentences into words using whitespace as the separator. Tokenization is applied for each
documents and special character is removed. The words are then filtered and stemming process is
followed. Filtering is removal of words which are of less im portance. The process of removing words
such as prepositions, articles and conjunctions are known as stop word filtering. Stemming algorithm is
used to convert different words form in to simple canonical form. It involves a set of procedure where all
wor ds with the same root are reduced with the same root to a common form. Porter stemmer method is
applied to improve the efficiency of data preprocessing . In this paper, algorithms of stop word removal
and stemming are used which mainly helps in improving th e accuracy of clustering and classification
techniques. In other words, the data preprocessing techniques helps in decrease of overall size of data set
in storage space and time