Mining Peculiar Compositions of Frequent Substrings from Sparse Text Data Using Background Texts
Abstract
We consider mining unusual patterns from text T . Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and composition s w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy , each of x and y is more frequent in B than in T , and conversely w = xy is more frequent in T . The frequency of xy in T is very small since x and y are infrequent in T , but xy is relatively abundant in T compared to xy in B . Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.
Cite
Text
Ikeda and Suzuki. "Mining Peculiar Compositions of Frequent Substrings from Sparse Text Data Using Background Texts." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2009. doi:10.1007/978-3-642-04180-8_56Markdown
[Ikeda and Suzuki. "Mining Peculiar Compositions of Frequent Substrings from Sparse Text Data Using Background Texts." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2009.](https://mlanthology.org/ecmlpkdd/2009/ikeda2009ecmlpkdd-mining/) doi:10.1007/978-3-642-04180-8_56BibTeX
@inproceedings{ikeda2009ecmlpkdd-mining,
title = {{Mining Peculiar Compositions of Frequent Substrings from Sparse Text Data Using Background Texts}},
author = {Ikeda, Daisuke and Suzuki, Einoshin},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2009},
pages = {596-611},
doi = {10.1007/978-3-642-04180-8_56},
url = {https://mlanthology.org/ecmlpkdd/2009/ikeda2009ecmlpkdd-mining/}
}