Using tweepy to access Twitters Streaming API

2024/11/15 6:08:26

I'm currently having trouble getting example code for using tweepy to access Twitter's Streaming API to run correctly (err...or at least how I expect it to run). I'm using a recent clone of tweepy from GitHub (labeled version 1.9) and Python 2.7.1.

I've tried example code from three sources, in each case using "twitter" as a test term for tracking:

  1. O'Rilley Answers code: How to Capture Tweets in Real-time with Twitter's Streaming API

  2. Andrew Robinson's blog: Using Tweepy to access the Twitter Stream

  3. Tweepy examples repository on GitHub (which, as Andrew Robinson has done, can be easily modified to support OAuth authentication): streamwatcher.py

In all three cases I get the same result: Authentication is successful, no errors are produced, and the main program loop seems to be executing w/o any problems. I see network usage jump to about 200KB/s, and the python process jumps to near 100% CPU usage, so I think data is being received. Nothing is output to the console, however.

I suspect that tweepy's Stream class is not calling the custom callback method for some reason. I've tried rewriting the callback methods in each example to produce output whenever they're called, which seems to confirm this. This is one very simple bit of test code based on Andrew Robinson's blog entry (with my app's keys removed, of course):

# -*- coding: utf-8 -*-import tweepyconsumer_key = ''
consumer_secret = ''access_token_key = ''
access_token_secret = ''auth1 = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth1.set_access_token(access_token_key, access_token_secret)class StreamListener(tweepy.StreamListener):def on_status(self, tweet):print 'Ran on_status'def on_error(self, status_code):print 'Error: ' + repr(status_code)return Falsedef on_data(self, data):print 'Ok, this is actually running'l = StreamListener()
streamer = tweepy.Stream(auth=auth1, listener=l)
#setTerms = ['hello', 'goodbye', 'goodnight', 'good morning']
setTerms = ['twitter']
streamer.filter(track = setTerms)

What am I doing wrong?

Answer

I ran into this as well and fixed it on my local checkout by changing line 160 in streaming.py to

if delimited_string.strip().isdigit():

This seems to be a known issue/bug in Tweepy - should have checked the issues list before doing all that debugging :) -

https://github.com/tweepy/tweepy/pull/173 https://github.com/tweepy/tweepy/pull/182

https://en.xdnf.cn/q/72273.html

Related Q&A

Jupyter notebook, how to run multiple cells simultaneously?

I defined a python function which run a bash script. Lets say the function is: calc(x,y,z). If I run this function in python with some variables,>>> calc(1,2,3)It generates a C code which simu…

How to make a slice of DataFrame and fillna in specific slice using Python Pandas?

The problem: let us take Titanic dataset from Kaggle. I have dataframe with columns "Pclass", "Sex" and "Age". I need to fill NaN in column "Age" with a median f…

Pythons difflib SequenceMatcher speed up

Im using difflib SequenceMatcher (ratio() method) to define similarity between text files. While difflib is relatively fast to compare a small set of text files e.g. 10 files of 70 kb on average compar…

create an asymmetric colormap

I am creating a colormap to map colors in a folium choropleth map, using code from here:from branca.colormap import linearcolormap = linear.RdBu.scale(df.MyValue.min(),df.MyValue.max())colormapAs you c…

NLTK - Get and Simplify List of Tags

Im using the Brown Corpus. I want some way to print out all the possible tags and their names (not just tag abbreviations). There are also quite a few tags, is there a way to simplify the tags? By sim…

PolynomialFeatures object has no attribute predict

I want to apply k-fold cross validation on the following regression models:Linear Regression Polynomial Regression Support Vector Regression Decision Tree Regression Random Forest RegressionI am able t…

Error module object has no attribute freetype

I am using this code Link but it displays error of module object has no attribute i tried to pip install freetype but nothing happened. Can anyone please guide me with this.import cv2 import numpy as …

Count total number of white pixels in an image

I am trying to count total number of white pixels in the following image:But with my code, I get this errorsrc is not a numpy array, neither a scalar.This is my code: img=cv2.imread(filename,1) TP= wid…

Pass a JSON object to an url with requests

So, I want to use Kenneth excellent requests module. Stumbled up this problem while trying to use the Freebase API.Basically, their API looks like that:https://www.googleapis.com/freebase/v1/mqlread?q…

jenkinsapi python - how to trigger and track the job result

I am using JenkinsAPI to trigger parametrized jobs. I am aware of the REST API that Jenkins use, but our setup does not allow that directly; so the main mean for me to trigger jobs is through this libr…