Abstract:
The term frequency { inverse document frequency (TF-IDF) weighting sch-
eme is widely used in text classi cation for weighting the features of the vector space model
(VSM). It aims at enhancing words' discriminating capabilities by weighing up the less
frequently used words and, at the same time, weighing down the high frequency words (i.e.,
the common words such as prepositions). This paper attempts to provide an enhanced
variant of the well-known TF-IDF method. The TF-IDF is a statistical estimation that
computes the weight of each word based on the frequency of the word in both the document
and the entire data collection. In this work, we propose considering the word's standard
deviation as another factor when computing the word's weight. That is, the common
words tend to have larger standard deviations more than the uncommon words. In other
words, the more the word appears in documents, the greater the standard deviation is.
To investigate the proposed TF-IDF based model, we conducted some experiments for
Arabic text classi cation. We used a training textual data collection that contains 1,750
documents of ve categories (250 documents for testing). The experimental results show
that the proposed approach is superior to the standard TF-IDF term weighting scheme.
Keywords: Arabic, Text, Classification, TF-IDF, Singular value decomposition.
