[Answer]-Python, handle persistent http connection in web application

by

1👍

✅

I did this recently for a project. You’ll need to run the stream consumer as a separate python process. It doesn’t need to be part of your Django application at all.

Basically I had:

from tweepy import OAuthHandler
from tweepy import Stream
from tweepy.streaming import StreamListener

from myproject.myapp.utils import do_something_with_tweet

class StdOutListener(StreamListener):

    def on_data(self, data):
        do_something_with_tweet(data)
        return True

def main():
    listener = StdOutListener()

    auth = OAuthHandler(
        TWITTER_CONSUMER_KEY,
        TWITTER_CONSUMER_SECRET)

    auth.set_access_token(
        TWITTER_ACCESS_TOKEN,
        TWITTER_ACCESS_SECRET)

    try:
        stream = Stream(auth, listener)
        stream.filter(track=['#something', ])
    except (KeyboardInterrupt, SystemExit):
        print 'Stopping Twitter Streaming Client'


if __name__ == '__main__':
    main()

This way you can run this as a separate process and pass the tweet data to some function to save it or whatever and Django can run happily elsewhere.

Plus points would be to use celery to process your tweet data in asynchronous tasks: https://celery.readthedocs.org

👤krak3n

Leave a comment