Blog 2 for Social Media Analytics

After attending so many classes, I know more about analytical methods for content and sentiment. Jaccard index can be used for comparing the similarity and diversity between two documents. During this process, we need to consider term weighting. That's how we can determine whether a word is important. Maybe we find one word appear frequently, So it's more important. This approach is not completely correct, because there may be part of the words which belongs to structure words. Although they are used frequently in a specific document, we can't categorize this document by means of these unrepresentative words.

A typical processing technology shortly named TF-IDF is an information retrieval and text mining techniques for common weighting. The TF-IDF value increases proportionally to the number of times a word appears in the document, but is offset by the frequency of the word in the corpus, which helps to control for the fact that some words are generally more common than others. TF-IDF weight calculation method and cosine similarity often are used together in the vector space model, in order to judge the similarity between two documents.

Text classification make our daily life more efficient, we don't need to process all the text with the same procedures. Naive Bayes Text Classification is a good way for classification. In addition, K-Means Clustering offer an iterative refinement technique to measure distance of the similar points.

In term of sentiment analysis, actually we are in touch with many things and express our opinions everyday. How do we evaluate one object, one thing? Or how do we make a decision when we need to choose. In most cases, we don't pay attention to sentiment analysis. However, it plays an important role in collecting information to make decisions in fact. Furthermore, opinions are divided into various categories according to different norms, such as direct opinion, comparative opinion, explicit opinion, implicit opinion and so on. Sentiment classification methods include dictionary-based approach and supervised learning. The former is to check the polarity scores of the words in sentences to be classified. The latter is analogous to text classification.

This course requires some Python programming. Unfortunately, I have almost no programming foundation. Now I begin to do some Python learning. It seem a little hard for me to remember so many language rules. If someone who read this blog can offer some ideas about how learning coding well to me, perhaps I will learn Python more effectively. I look forward to mastering Python and do some application with it. Thank you for reading this blog. Please give some comments if you like. See you next time.