Iterators

Sometimes you need to make several requests to the same API endpoint in order to get all the data you want (e.g. getting more than 200 tweets of an user). Some iterators are included in Peony and usable through the peony.iterators module that deals with the actual iteration, getting all the responses you need.

Cursor iterators

This is an iterator for endpoints using the cursor parameter (e.g. followers/ids.json). The first argument given to the iterator is the coroutine function that will make the request.

from peony import PeonyClient

# creds being a dictionnary containing your api keys
client = PeonyClient(**creds)

async def get_followers(user_id, **additional_params):
    request = client.api.followers.ids.get(id=user_id, count=5000,
                                           **additional_params)
    followers_ids = request.iterator.with_cursor()

    followers = []
    async for data in followers_ids:
        followers.extend(data.ids)

    return followers

Max_id iterators

An iterator for endpoints using the max_id parameter (e.g. statuses/user_timeline.json):

from peony import PeonyClient

client = PeonyClient(**creds)

async def get_tweets(user_id, n_tweets=1600, **additional_params):
    request = client.api.statuses.user_timeline.get(user_id=user_id,
                                                    count=200,
                                                    **additional_params)
    responses = request.iterator.with_max_id()

    user_tweets = []

    async for tweets in responses:
        user_tweets.extend(tweets)

        if len(user_tweets) >= n_tweets:
            user_tweets = user_tweets[:n_tweets]
            break

    return user_tweets

Since_id iterators

An iterator for endpoints using the since_id parameter (e.g. GET statuses/home_timeline.json):

import asyncio
import html

from peony import PeonyClient

client = peony.PeonyClient(**creds)

async def get_home(since_id=None, **params):
    request = client.api.statuses.home_timeline.get(count=200, **params)
    responses = request.iterator.with_since_id()

    home = []
    async for tweets in responses:
        for tweet in reversed(tweets):
            text = html.unescape(tweet.text)
            print("@{user.screen_name}: {text}".format(user=tweet.user,
                                                       text=text))
            print("-"*10)

        await asyncio.sleep(120)

    return sorted(home, key=lambda tweet: tweet.id)

Note

with_since_id() has a fill_gaps parameter that will try to find all the tweets that were sent between 2 iterations if it cannot be found in a single request (more than 200 tweets were sent)

responses = request.iterator.with_since_id(fill_gaps=True)

Note

Both with_since_id() and with_max_id() have a force parameter that can be used in case you need to keep making requests after a request returned no content. Set force to True if this is the case.