Live twitter sentiment analysis based on a given search term
-
You'll need to install the following libraries in a conda env or venv or globally on your PC
-- tweepy for scraping tweets
-- afinn for getting the sentiments
-- confluent_kafka to stream tweets to kafka
-- socket for kafka connection -
Create a twitter developer account and get and paste your bearer_token where it is written YOUR_BEARER_TOKEN_GOES_HERE in stream_tweets.py and replace the search term accordingly.
-
Run zookeeper command: bin/zookeeper-server-start.sh config/zookeeper.properties
-
Start Kafka after a few seconds, once zookeeper is up and running using the command: bin/kafka-server-start.sh config/server.properties
-
Create a kafka topic named tweet-polarizer
-
Configure logstash pipeline as per logstash.config file and use the config as: bin/logstash -f config/logstash.conf
-
Create elastic search index named search-tweet_polarizer
-
I have used Elastic Cloud to save and view the data, you may configure it and add your cloud key in logstash config.
-
Run stream_tweets.py and visualize the data as per the requirement.