ORCID

Jeffrey Stanton: 0000-0001-6120-7273

Document Type

Article

Date

Fall 10-1-2020

Keywords

data analysis, text analysis, word embedding

Language

English

Disciplines

Applied Linguistics | Databases and Information Systems | Library and Information Science | Management Sciences and Quantitative Methods

Description/Abstract

Researchers from many fields have used statistical tools to make sense of large bodies of text. Many tools support quantitative analysis of documents within a corpus, but relatively few studies have examined statistical characteristics of whole corpora. Statistical summaries of whole corpora and comparisons between corpora have potential application in the analysis of topically organized applications such social media platforms. In this study, we created matrix representations of several corpora and examined several statistical tests to make comparisons between pairs of corpora with respect to the topical homogeneity of documents within each corpus. Results of three experiments suggested that a matrix of cosine distances calculated from vector summaries of short phrases contains useful information about how closely the documents within a corpus relate to one another. Both the tested summarization method and a non-parametric test for comparing cosine distance matrices appear to have utility for examining and comparing corpora containing brief texts.

Source

submission

Creative Commons License

Creative Commons Attribution-No Derivative Works 4.0 International License
This work is licensed under a Creative Commons Attribution-No Derivative Works 4.0 International License.

Available for download on Friday, October 21, 2022

Share

COinS