Extract & Load Stripe.com data

If your app uses Stripe to handle credit card payments, you likely want to analyze that data. You probably also want to combine this payment data with other operational data (CRM systems, Google Analytics, etc.) to further analyze your business. Whether it’s building predictive models or visualizing this data in a dashboard, somehow, you need to land your Stripe data in a data warehouse.

Our Simple Data Pipe app makes it easy for you to extract data from the source you choose and load it into IBM’s dashDB cloud data warehouse service, where you can combine it with other data sets and perform analysis. This article explains how to set up Simple Data Pipe to extract your Stripe data and load it into dashDB.

Deploy Simple Data Pipe

The Simple Data Pipe app runs on Bluemix, IBM’s cloud application platform service. The fastest way to deploy this app to Bluemix is to click this Deploy to Bluemix button—it automatically creates a web-based version of the app with all the services you need: Node.js, Cloudant, dashDB, DataWorks, and SSO.

If you prefer instead to deploy manually, please refer to the readme.

On the Deploy to Bluemix screen, give the app a name that’s meaningful to you, like Simple Data Pipe, and click Save. (If you have any trouble deploying, make sure you haven’t already reached your Bluemix memory and services quotas. You get 10 with your account’s free trial period.)

To launch the app, click the View Your App button. Your Simple Data Pipe app opens, but you’re not ready to use it yet. Leave this browser window open. You’ll return to it in a few minutes.

Grant Simple Data Pipe Access to Your Stripe Account

In Stripe, you must grant Simple Data Pipe read-only access to your data. To do so:

  1. Open the stripe.com dashboard.
  2. Log in using your stripe.com credentials.

    If you don’t yet have an account, sign up. You can try these steps with test data. You can create test data through the UI for the Stripe test environment. See their docs for test card numbers and other simulated parameters.

  3. On the upper right of the screen, click Your account > Account Settings.
  4. Open your stripe.com account settings

  5. Click the API Keys tab.
  6. Open the API Keys tab

  7. Note the key value for the data you want to move. You’ll need this string in the next section.
    • To pipe only test data, copy the value in Test Secret Key.
    • To pipe live production data, copy the value in Live Secret Key.

    stripe_acct_keys

  8. Click the Connect tab.
  9. Register your platform.
    • If you never connected to an app before, go to the lower left of the screen and click Register your platform.
      register_none
    • If you have previously connected to an app, click the Platform Settings button, then Register your platform registerplat1
  10. In the standalone accounts section enter the following information:
    • Name of the app. Enter any unique identifier, like My Simple Data Pipe Demo App.
    • Website URL (used for reference only) of your Simple Data Pipe application. You can get this URL from the Simple Data Pipe browser window you left open. Just copy and paste the URL from the address bar. It is: https://<your_data_pipes_host_name>.mybluemix.net
    • Redirect URIs. Enter a value for the Stripe environment in which you’re working: development or production. Your entry will be the same for either one: https://<your_data_pipes_host_name>.mybluemix.net/authCallback

    app_info

  11. Copy the client_id value the environment you’re working in: either production or development. You’ll need this string in minute to set up your Data Pipe run.
  12. Click Done.

Run a Data Pipe Job to Load Data from Stripe

  1. Return to the Simple Data Pipe app.

    If you can’t find the Simple Data Pipes app browser window you left open, find the app’s route through your Bluemix dashboard.

  2. Choose Create A New Pipe.
  3. Under Type choose Stripe.
  4. Enter a unique Name and Description.
    Create a new stripe pipe
  5. Click Proceed to create the new pipe.
  6. In the Connect tab enter the following values:
    • In Consumer key, enter the client_id value you copied from Stripe.
    • In Consumer secret, enter the corresponding Test or Live Secret Key you noted earlier in your Stripe account settings.

    Specify stripe.com oauth credentials

  7. Click Connect to Stripe to start the authorization process.
  8. If prompted, enter your stripe user credentials.
  9. If you have not activated your account yet, the following form is displayed prompting you to accept payments with Stripe. If you are planning to pull only data from your test environment, do not fill it out. Instead, go the upper right and click Skip this account form.
    skip_acctform
  10. You will be redirected to Simple Data Pipe. Click Save and continue.

    Note: After you complete this step, your Simple Data Pipe has read-only access to all data in the Stripe environment you selected (test or production). This is reflected your Stripe account settings.

    Save stripe.com oauth credentials

  11. On the Pick tables tab, choose the table(s) you want to extract from Stripe and click save and continue.
    Select the stripe object to copy
  12. If you want to run the pipe immediately, on the Schedule tab, click the Skip button.

    If you prefer to schedule a run, enter a time to run the pipe and click Save and continue.

    Note: You can run only one pipe job at a time.

    Schedule the Stripe pipe run

  13. If you skipped scheduling, on the Activity tab, click Run Now.
  14. Wait for the data transfer to complete. This will take several minutes.
    Monitor stripe pipe progress

    Tip: If you want to peek under the hood you can read how to review the DataWorks activity processing status.

  15. After data copy completes, click View details to open the pipe run summary and Details next to the pipe run for more information.
    Review pipe run results

Delete a Data Pipe

You can delete a pipe run by clicking on X next to the pipe name.

Deleting a pipe

Revoke Simple Data Pipe Access to Your Stripe Data

You can revoke Simple Data Pipe’s read-only access to your Stripe data at any time by taking the following steps.

  1. Open the stripe.com dashboard.
  2. Log in using your stripe.com credentials.
  3. Open your account settings (Your account > Account Settings).
  4. Select the Connect tab and Connected Applications.
  5. Click Revoke access.
    Revoke Simple Data Pipe's access to your billing information

That’s it! Stay tuned for our next article, where we’ll connect a dashboard to visualize our Stripe data.

Join The Discussion

Your email address will not be published. Required fields are marked *