Polling for Activities using REST API

November 20, 2015 | by

Earlier this year we published this post that described how to poll for lead changes within Marketo.  This post is similar, but this time we’ll poll for activities.

Activities are a core object in the Marketo Platform. Activities are the behavioral data that is stored about every webpage visit, email open, webinar attendance, or trade show attendance.  A common use case is combining activity data with data from other parts of an organization.

This sample program contains 6 steps:

  1. Call Get Lead Activities to generate a list of all activity records created since a given date/time. We use a filter to limit the type of activity records that are returned.
  2. Extract fields of interest from each activity record.
  3. Call Get Multiple Leads by Filter Type to generate a list of lead records that correspond with the activities from Step 1. We use the leadId field extracted from activity record in Step 2 as our filter to specify which leads are returned.
  4. Extract fields of interest from each lead record.
  5. Join the activity data from step 2 with the lead data from Step 4.
  6. Transform the data from Step 5 into a format that is consumable by an external system.

As the diagram below shows, for this example we have chosen to capture email-related activities.

Activity_Sync_Overview
Program Input

By default the program goes back in time one day from the current date to look for changes.  So you could run this program at the same time each day for example.  To go farther back in time, you can specify the number of days as a command line argument, effectively increasing the window of time.

The program contains several variables that you can modify:

CUSTOM_SERVICE_DATA – This contains your Marketo Custom Service data (Account Id, Client Id, Client Secret).

READ_BATCH_SIZE – This is the number of records to retrieve at a time.  Use this to tune the response body size.

LEAD_FIELDS – This contains a list of lead fields that we want to collect.

ACTIVITY_TYPES – This contains a list of activity types that we want to collect.

Program Logic

First we establish our time window, compose our REST endpoint URLS, and obtain our Authentication access token.  Next we fire up a Get Paging Token/Get Lead Activities loop which runs until we exhaust the supply of activities.  The purpose of this loop is to retrieve activity records, and extract fields of interest from those records.

We tell Get Lead Activities to look only for the following activity types:

  • Email Delivered
  • Unsubscribe Email
  • Open Email
  • Click Email.

We extract the following fields of interest from each activity record:

  • leadId
  • activityType
  • activityDate
  • primaryAttributeValue

You are free to select any combination of activity types and activity fields to suit your purpose.

Next we fire up a Get Multiple Leads by Filter Type loop which runs until we exhaust the supply of leads.  Note that we use the “filterType=id” parameter in combination with a series of “filterValues” parameters to limit the lead records retrieved to only those leads associated with activities that we retrieved earlier.

We extract the following fields of interest from each lead record:

  • firstName
  • lastName
  • email

Again, you are feel to select any lead fields that you like.

Next we join the lead fields with the activity fields by using the lead id to link them together.

Finally, we loop through all of the data, transform it into JSON format, and print it to the console.

Program Output

Here is an example of output from the sample program.  This shows the activity fields and the lead fields combined together as JSON “result” objects.  The idea here is that you could pass this JSON as a request payload to an external web service.

Program Code

That’s it.  Happy coding!