10

Demo : Twitter streaming and sentiment analysis using Kafka, OCS, MongoDB &...

 2 years ago
source link: https://dev.to/ksingh7/demo-twitter-streaming-and-sentiment-analysis-using-kafka-ocs-mongodb-openshift-kubernetes-161f
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Cover image for Demo : Twitter streaming and sentiment analysis using Kafka, OCS, MongoDB & OpenShift (Kubernetes)
karan singh

Posted on Feb 15

Demo : Twitter streaming and sentiment analysis using Kafka, OCS, MongoDB & OpenShift (Kubernetes)

You know tech tools are cool, but unless you have a defined use case it's hard to put things into perspective and understand how different tools can interact with each other, help solve a problem or explore new use cases.

So to educate and motivate our technical buyers, sellers and customers, I created a fancy use case of ingesting live Twitter tweets and applying sentiment analysis to it. For this demo, i used the following tools

  • Twitter API : Realtime streaming data source

  • Red Hat AMQ Streams : Apache Kafka cluster to store real-time streaming data coming into the system

  • MongoDB : Storing tweets for long term persistence from Kafka into a schema-less NoSQL database

  • Red Hat OpenShift Container Storage : Used for providing RWO (in this project), RWX, Object Storage persistence storage for Kafka and MongoDB apps running on OpenShift

  • *Red Hat OpenShift Container Platform *: Enterprise grade k8s distribution for container apps

  • **Aylien : **Sentiment analysis solution backend

  • Python : Backend API app to trigger data sourcing from twitter, move data from Kafka to MongoDB, server data to frontend app

  • Frontend : basic HTLM, CSS, Javascript-based frontend to plot some graphs

This slide deck should give you a glimpse of how the demo would look like (demo youtube/github link below)

Slide Deck

And here is the actual demo recording that you can go through, where i have explained how these components work together and make this a viable solution if you have a real-world use case along the same lines

YouTube Video Link]

If you are interested in running this demo by yourself, you can find the code in my repo, Github project link : https://github.com/ksingh7/twitter_streaming_app_on_openshift_OCS

Happy Analysing Live Tweets


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK