Tweet Sentiment Heatmap
I was always fascinated with cloud computing and working with cloud services. That is the reason I took the Cloud Computing class while pursuing my Masters in Computer Science at New York University. Here is a demo for one of the projects I worked on using a bunch of Amazon Web Services, Node.js, and socket.io.
Twitter Stream Listener
We have a Twitter Stream Listener backend service that listens to random samples of all public tweets and sends them to AWS SQS.
Sentiment analyzer is a backend service that retrieves the tweets from SQS and runs sentiment analysis through IBM Watson’s Alchemy API. It then publishes the tweet with sentiment information to an AWS SNS topic.
A node server subscribes to the AWS SNS topic and also serves our front-end application page. It uses socket.io to transmit the tweet and sentiment information to all the browser clients connected via web sockets.
Sequence of events
- The program listens to random sample of all public statuses using twitter streaming API
- It adds the tweets containing location information to Amazon SQS for processing
- Another thread periodically retrieves these tweets from SQS for sentiment analysis on the text of these tweets
- It uses Alchemy API for sentiment analysis
- It publishes the tweet’s text and location along with the sentiment to all the endpoints subscribed to AWS SNS topic
- Node server receives the tweet and sentiment via webhooks from AWS SNS topic and transmits it to web clients using socket.io
- The SPA (Single Page Application) on user’s browser (served from node server) connects to the node server through web sockets and updates the heat map by using tweet’s latitude/longitude and sentiment score.
Real-time Twitter sentiment heatmap