Connecting to Twitter with Data Cube

You can use Data Cube to connect to the data on Twitter using the Twitter public API. You can crawl tweets by specific Twitter users, or you can crawl tweets that contain specific hashtags and keywords. After the data source is added, you can restructure the data. For example, you can combine data from two or more data sources into a single data source. For information about the available data restructuring options, see Restructuring Data in Data Cube.

Before You Begin

You must sign in to a Twitter account to use the Twitter connector.

Procedure

1. From the Data Cube dashboard, next to Twitter, click Add New.

   Tip

   Alternatively, click Twitter to open the Data Sources (Twitter) page, and then click Add Twitter in the upper-right corner.

   The New Data Source (Twitter) page appears.

2. In the Data Source Name section, enter the following information:

3. Field Name	Description	Steps
   Index Server	The Index Server entity that you want to use for the data source. The Index Server list is populated by the Index Servers configured for Data Cube in your CommCell Console.	Click the Index Server list and select an Index Server.
   Data Source Name	The name of the data source as it will appear in the Data Cube dashboard.	Enter a name for the data source. Note: Only alphanumeric, dash, and underscore characters are supported.
   Data Source Description	A short description of the data source that is visible to any user who can view the data source in Data Cube.	Optional: Enter a description for the data source.

4. Click Next.

5. In the Login section, proceed as follows:

   1. Click Sign In .

      Tip

      The login page is a pop-up. If you do not see the login page after you click Sign In, try allowing pop-ups in your browser.

   2. In the Twitter login pop-up that appears, enter your Twitter account credentials and login.

      Note

      If it is your first time using the Twitter connector with your Twitter account, click Authorize App to allow the connector to use your account.

      If the login is successful, under Sign In, the message You are logged in Twitter_handle appears.

6. Click Next.

7. Under Crawl Specifications, select the type of crawl that you want to perform.

   • Crawl tweets by specific Twitter users.

     For Crawl Type, select User, and then enter the following information:

     Field Name	Description	Steps
     Username	The username, or Twitter handle, is the specific Twitter user or users that you want to crawl. Only tweets from these users are crawled for the data source.	Enter a comma-separated list of Twitter usernames.
     Date Range	You can limit the crawl to collect tweets from a specific date range.	Optional: Click the box and use the calendar widget to select a beginning and end date for the tweets.
     Hash tags	You can limit the crawl to tweets that contain one or more hash tags.	Optional: Enter a comma-separated list of hash tags.
     Incremental Crawl	When selected, each crawl will only append new data to the data source. If not selected, the data source is overwritten with each crawl.	Optional: Click the slider to enable incremental crawling.

   • Crawl tweets by keywords or hashtags.

     For Crawl Type, select Live Streaming, and then enter the following information:

     Field Name	Description	Steps
     Streaming keyword	The keywords and hashtags that you want to crawl. Only tweets that have the keywords or hashtags in this list are included in the data source.	Type a keyword or hashtag, and then press Enter.
     Trending Topics	The Trending Topics table contains keywords and hash tags that are trending on Twitter.	Click a keyword or hashtag to add it to the Streaming Keyword list.

8. Click Submit.

   The data source name Configuration page appears.

What to Do Next

Crawling a Data Source in Data Cube