Twitter Input Adapter

Contents

Introduction
Stream Types
Adapter Properties
Control Port
Obtaining Twitter OAuth Tokens
Related Topics

Introduction

The TIBCO StreamBase® Twitter Input adapter connects a StreamBase application to the web service provided by Twitter, Inc. It can receive Twitter status messages and direct messages, commonly called tweets.

The input adapter can be initially set up to respond to control messages for changing at runtime the stream being watched. It can watch the user, sample, link, and filter twitter streams, and can perform searches. The adapter can also process direct messages on the user stream, if so enabled.

By default, the input adapter is configured to not connect to any twitter streams on startup, awaiting a control message to perform an operation.

The adapter must be configured with the correct OAuth tokens for it to perform any operations. The OAuth can be set up in the security tab, as described in Obtaining Twitter OAuth Tokens below.

The adapter has samples, described in Twitter Adapter Samples.

Fields in schemas used by the Twitter adapters correspond to attributes in JSON-encoded tweets accessed via Twitter4J library calls. The library is provided with StreamBase. To help you understand what fields in schemas represent, the following table summarizes Twitter objects and attributes, and provides links to their documentation at dev.twitter.com.

Object Example Attributes Reference
Tweet id, following, followers_count, geo_enabled, status Platform Objects:Tweets
User id, following, followers_count, geo_enabled, status Platform Objects:Users
Geo Place country_code, locality, region, street_address, postal_code Platform Objects:Places
Entity hashtags, media, urls, user_mentions Platform Objects:Entities

Briefly,

  • Tweets are the basic atomic building block of all things Twitter. Users tweet Tweets, also known more generically as status updates. Tweets can be embedded, replied to, favorited, unfavorited and deleted.

  • Users can be anyone or anything. They tweet, follow, create lists, have a home_timeline, can be mentioned, and can be looked up in bulk.

  • Places are specific, named locations with corresponding geo coordinates. They can be attached to Tweets by specifying a place_id when tweeting.

  • Entities provide metadata and contextual information about content posted on Twitter and are not autonomous objects.

Schemas for the StreamBase Twitter adapters encode a large subset of attributes for these objects. You can modify the schemas to include others.

Stream Types

The input adapter has multiple twitter stream types it can handle. Each separate input adapter can handle one of the following at a time:

  • Filter — Start consuming public statuses that match one or more keywords or user IDs. At least one keyword or user ID must be specified. Multiple parameters may be specified, allowing most clients to use a single connection to the Streaming API. The default access level allows tracking up to 200 keywords, 400 follow user IDs, and ten 1-degree location boxes. Increased access levels allow 80,000 follow user IDs ("shadow" role), 400,000 follow user IDs ("birddog" role), 10,000 track keywords ("restricted track" role), 200,000 track keywords ("partner track" role), and 200 10-degree location boxes ("locRestricted" role). Increased track access levels also pass a higher proportion of statuses before limiting the stream.

    Note

    To increase your access level (to gardenhose or firehose levels), you must make arrangements with Twitter, Inc. and possibly with one of its resellers. Contact the Twitter Certified Products Program, and also consult its Developer Rules of the Road page.

  • Sample — Starts listening to a random sample of all public statuses. The default access level provides a small proportion (1% or less) of the Firehose. The gardenhose access level provides a proportion more suitable for data mining and research applications that desire a larger proportion to make up a statistically significant sample.

  • Link — Starts listening on the link feed, which publishes all status updates that contain Web links.

  • User — User Streams provide a stream of data and events specific to the authenticated user and provide real-time updates of all data needed to update a desktop application display of the current user's feed, with almost no rate limits or latency. This feed also produces direct messages.

Adapter Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.

General Tab

Name: Use this field to specify or change the component's name, which must be unique in the application. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class: A field that shows the fully qualified class name that implements the functionality of this adapter. Use this class name when loading the adapter in StreamSQL programs with the APPLY JAVA statement. You can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start with application: If this field is set to Yes or to a module parameter that evaluates to true, an instance of this adapter starts as part of the containing StreamBase Server. If this field is set to No or to a module parameter that evaluates to false, the adapter is loaded with the server, but does not start until you send an sbadmin resume command, or until you start the component with StreamBase Manager. With this option set to No or false, the adapter does not start even if the application as a whole is suspended and later resumed. The recommended setting is selected by default.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports and Error Streams to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property Description
Enable Direct Messages Port Enabling this port allows your application to receive tweets directly from users. Your API key must be have appropriate security to allow this feature.
Enable Users Port The users output port is used for the SearchUsers, Followers, and Following commands.
Enable Places Port The places output port is used for SearchPlaces commands.
Enable Friendships Port The friendships output port is used for LookupFriendship commands.
Enable Trends Port The trends output port is used for SearchTrends commands.
Enable Rate Limits Status Port The rate limits status output port is used for RateLimitsStatus commands.
Enable Status Port The status output port emits status messages for your application's consumption.
Startup Twitter Stream The initial stream to connect to when the application is started.
Keywords The keywords to use when the start up type is set up to filter.
User Ids The user IDs to use when the start up type is set up to filter.
Log Level Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE, and ALL.

Security Tab

Property Type Description
Twitter Consumer Key string Obtain Consumer Key from app @ developer.twitter.com
Twitter Consumer Secret string Obtain Consumer Secret from app @ developer.twitter.com
Twitter Access Token Secret string Obtain Access Token Secret from app @ developer.twitter.com
Twitter Access Token string Obtain Access Token from app @ developer.twitter.com

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Control Port

Description

The control port is used to send action commands to the adapter.

Control Port Schema

  • Command, string. The command to send to the adapter. Specify one of the following:

    • Filter — This command tells the adapter to listen to a filter stream based on the criteria given in the Filter tuple.

    • Sample — This command tells the adapter to start listening to the sample twitter stream.

    • Link — This command tells the adapter to start listening to the link twitter stream. This stream needs special permission from Twitter, Inc. to use.

    • User — This command tells the adapter to start listening to the current users twitter stream and to direct messages, if enabled.

    • SearchTweets — This command performs a search of all tweets based on the criteria given in the SearchTweet tuple.

    • SearchUsers — This command performs a search of users based on the criteria given in the SearchUsers tuple.

    • SearchPlaces — This command performs a search of place based on the criteria given in the SearchPlaces tuple.

    • SearchTrends — This command performs a search for trends in a geographic location using the SearchTrends tuple.

    • Timeline — This command tries to receive as many historical status updates from the current user's own timeline as it can.

    • LookupFriendship — This command looks up the friendship relationship between the current user and the specified ScreenName.

    • Follow — Start following a particular user.

    • Unfollow — Stop following a particular user.

    • Followers — List the current user's followers.

    • Following — List the users that the current user is following.

    • RateLimitsStatus — Outputs all the current user's rate limits to determine whether we can perform more operations.

    • Disconnect — tells the adapter to disconnect from the current twitter stream.

  • Filter, Tuple. The tuple used to create the filter stream. This field is only used when the Command is filter, and is set to one of:

    • Count, int. The number of tweets to go back to in history and fetch, before starting to stream.

    • UserIDs, list(long), (optional). A list of user IDs to add to this stream.

    • Keywords, list(string), (optional). A list of keywords to use when creating this stream.

  • SearchTweets, Tuple, the tuple used to search. This field is only used when the Command is SearchTweets.

    • SearchTerm, string. The text to search tweets for.

    • Count, int. The maximum number of results to return. Please note that Twitter has its own maximum that may be less than this number.

    • Since, timestamp. The starting date to look for tweets.

    • GeoLocation, Tuple. The geo location to search for tweets.

      • Longitude, double. The longitude point to search.

      • Latitude, double. The latitude point to search.

      • Radius, double. The radius search starting from the position specified by the longitude and latitude.

      • Units, string. The units to use for the radius: km for kilometer, mi for miles.

  • SearchUsers, Tuple. The tuple used to search for users. This field is only used when Command is SearchUsers.

    • SearchTerm, string. The text to search tweets for.

  • SearchPlaces, Tuple, the tuple used to search for locations. This field is only used when Command is SearchPlaces.

    Either IP or Longitude and Latitude can be specified but only one will be used.

    • Longitude, double. The latitude to search near. This parameter is ignored unless it is inside the range -90.0 to +90.0 (North is positive), inclusive. It is also ignored if there is no corresponding Latitude parameter.

    • Latitude, double, The longitude to search near. The valid range for longitude is -180.0 to +180.0 (East is positive), inclusive. This parameter is ignored if outside that range, if it is not a number, or if there is no corresponding Longitude parameter.

    • IP, string. An IP address used when attempting to assign geolocation based on the user's IP address. Example Values: 74.125.19.104

    • Granularity, string. This is the smallest region to return, and must be one of:

      • poi

      • neighborhood

      • city

      • admin

      • country

      If no granularity is specified for the request, neighborhood is assumed. Specifying a granularity implies accepting all larger geographical units. For example, if set to city, results include places of type city, admin, and country.

    • Accuracy, string. A hint on the region in which to search. If a number, then this specifies a radius in meters. It can also be a string that must be a number suffixed with ft to specify feet, such as 100ft. If Accuracy is not specified, it defaults to 0m. In practice, accuracy is whatever accuracy the user's device is capable of measuring at its current location.

  • ScreenName, String. The screen name to use with the Command Follow or Unfollow.

  • SearchTrends, Tuple. The tuple used to search for trends. This field is only used when Command is SearchTrends.

    • GeoLocation (Both fields must have valid values.)

      • Longitude, double. The latitude to search near. The valid range is -90.0 to +90.0 (North is positive), inclusive.

      • Latitude, double. The longitude to search near. The valid range is -180.0 to +180.0 (East is positive), inclusive.

Obtaining Twitter OAuth Tokens

The Twitter adapter must be configured with valid Twitter OAuth consumer and access tokens, obtained without charge from Twitter's developer web site. Twitter OAuth tokens encapsulate the login credentials for one Twitter account. The tokens encapsulate your Twitter account's authentication rights without storing or requiring your account's password, and thereby remain effective even when you change your Twitter password. For additional details, see the Twitter OAuth documentation.

Follow the steps below to obtain OAuth tokens from Twitter.

  1. Decide in advance which Twitter account you will use for communication with your StreamBase application. You probably do not want to use a personal account for business communication, so set up a new business Twitter account if one does not already exist.

  2. Proceed to http://dev.twitter.com and use the Sign in link to log in with the designated Twitter account.

  3. Your Twitter account name is now displayed where the sign-in link was. Hover over it to display a context menu and select My Applications.

  4. Click the Create a new application link. This opens a form titled Create an Application.

    • Fill in a Name. This name must be unique for the logged-in Twitter account.

    • Fill in the Description field. Twitter requires at least 10 characters in this field.

    • Provide a URL in the Website field.

    • For this example, leave the Callback URL blank.

    • Accept the Develop Rules Of The Road by checking the Yes, I agree box.

    • Provide the CAPTCHA response.

    • Click the Create your Twitter application button.

  5. The application has been created. You will see the screen shown below. Click the Settings tab.

  6. Under the Application Type section select the Read and Write radio button (Note: if you plan to send and receive direct messages with your application, you must instead select Read, Write and Access direct messages). This allows your application to receive and post tweets. Click Update this Twitter application's settings to complete the update.

  7. Click the Details tab.

  8. At the bottom of the page click Create my access token tab.

  9. Copy, label, and save the Consumer Key , Consumer Secret , Access Token and Access Token Secret strings provided on this page. You will need to enter these tokens in your application's sbd.sbconf file.

Related Topics