Bulk Activity Extract

Bulk Activity Extract Endpoint Reference

The Bulk Activity Extract set of REST APIs provides a programmatic interface for retrieving large amounts of activity data out of Marketo.  For cases which do not require low latency, and need to transfer significant volumes of activity data out of Marketo, such as CRM-integration, ETL, data warehousing, and data archiving.

Permissions


The Bulk Activity Extract APIs requires that the owning API user have the Read-Only Activity or Read-Write Activity permissions.

Filters

Filter Type Data Type Required Notes
 createdAt Date Range  Yes Accepts a JSON object with the members startAt and endAt.  startAt accepts a datetime representing the low-watermark, and endAt accepts a datetime representing the high-watermark.  The range must be 31 days or fewer.

Jobs with this filter type will return all accessible records which were created within the date range.

Datetimes should be in an ISO-8601 format, without miliseconds.

 activityTypeIds Array[Integer] No Accepts a JSON object with one member, activityTypeIds.  The value must be an array of integers, corresponding to the desired activity types.

The “Delete Lead” activity is not supported (use Get Deleted Leads endpoint instead).

Retrieve activity type ids using the Get Activity Types endpoint.

primaryAttributeValueIds Array[Integer] No Accepts a JSON object with one member, primaryAttributeValueIds.  The value is an array of ids which specify the primary attributes to filter on.  A maximum of 50 ids may be specified.

The ids are the unique identifier for either a lead field or an asset, and can be retrieved by calling the appropriate REST API endpoint.  For example, to filter on a specific Form for the “Fill Out Form” activity, pass the Form name to Get Form by Name endpoint to retrieve the Form Id.

The following is a list of activity types where primary attribute filtering is supported.

Activity Type Primary Attribute  Value Id Retrieval Endpoint Asset Group
Change Data Value Lead field id Describe Lead Attribute Name
Change Score Lead field id Describe Lead Attribute Name
Change Status in Progression Program id Get Program by Name Marketing Program
Add to List Static list id Get Static List by Name Static List
Remove from List Static list id Get Static List by Name Static List
Fill Out Form Form id Get Form by Name Web Form

When using primaryAttributeValueIds, the activityTypeIds filter must be present and only contain activity ids that match the corresponding asset group.

Example:

For example, if you are filtering on Web Form assets, only the “Fill Out Form” activity type id is allowed in activityTypeIds.

Example Request Body:

   "filter": { 
      "createdAt": { 
         "startAt": "2021-07-01T23:59:59-00:00",
         "endAt": "2021-07-02T23:59:59-00:00"
      },
      "activityTypeIds": [ 
         2
      ],
      "primaryAttributeValueIds" : [
          16,102,95,8
      ]      
   }
}

primaryAttributeValueIds and primaryAttributeValues cannot be used together.

primaryAttributeValues
Array[String] No Accepts a JSON object with one member, primaryAttributeValues.  The value is an array of names which specify the primary attributes to filter on.  A maximum of 50 names may be specified.

The names are the unique identifier for either a lead field or an asset, and can be retrieved by calling the appropriate REST API endpoint.  For example, to filter on a specific Form for the “Fill Out Form” activity, pass the Form Id to Get Form by Id endpoint to retrieve the Form name.

The following is a list of activity types where primary attribute filtering is supported. 

Activity Type Primary Attribute Value Retrieval Endpoint Asset Group
Change Data Value Lead field displayName Describe Lead Attribute Name
Change Score Lead field displayName Describe Lead Attribute Name
Change Status in Progression Program name Get Program by Id Marketing Program
Add to List Static list name Get Static List by Id Static List
Remove from List Static list name Get Static List by Id Static List
Fill Out Form Form name Get Form by Id Web Form

Note that you must use “<program>.<asset>” notation to uniquely specify the name for the following asset groups: Marketing Program, Static List, Web Form.

Example:

For example, a form with name “MPS Outbound” that resides underneath program with name “GL_OP_ALL_2021” would be specified as “GL_OP_ALL_2021.MPS Outbound”.

Example Request Body:

   "filter": { 
      "createdAt": { 
         "startAt": "2021-07-01T23:59:59-00:00",
         "endAt": "2021-07-02T23:59:59-00:00"
      },
      "activityTypeIds": [ 
         2
      ],
      "primaryAttributeValues" : [
          "GL_OP_ALL_2021.MPS Outbound"
      ]      
   }
}

When using primaryAttributeValues, the activityTypeIds filter must be present and only contain activity ids that match the corresponding asset group.  For example, if you are filtering on Web Form assets, then only the “Fill Out Form” activity type id is allowed in activityTypeIds.

primaryAttributeValues and primaryAttributeValueIds cannot be used together.

Options

Parameter Data Type Required Notes
filter Array[Object] Yes Accepts an array of filters.  Exactly one createdAt filter must be included in the array.  An optional activityTypeIds filter may be included.

The filters are applied to the accessible activity set, and the resulting set of activities is returned by the export job.

format String No Accepts one of: CSV, TSV, SSV  The exported file will be rendered as a comma-separated values, tab-separated values, or space-separated values file, respectively if set.

Defaults to CSV if unset.

 columnHeaderNames Object No A JSON object containing key-value pairs of field and column header names. The key must be the name of a field included in the export job. The value will be the name of the exported column header for that field.
fields Array[String] No Optional array of strings containing field values.  The listed fields will be included in the exported file.  By default, the following fields are returned:

    • marketoGUID
    • leadId
    • activityDate
    • activityTypeId
    • campaignId
    • primaryAttributeValueId
    • primaryAttributeValue
    • attributes

This parameter can be used to reduce the number of fields that are returned by specifying a subset from the list above.

Example:

“fields”: [“leadId”, “activityDate”, “activityTypeId”]

Creating a Job

To export records, you first need to define the job and the set of records that you want to retrieve.  Create the job using the Create Export Activity Job endpoint.  When exporting activities there are two primary filters which can be applied: createdAt, which is always required, and activityTypeIds, which is optional.  The createdAt filter is used to define a date range in which activities were created, using the startAt and endAt parameters, both of which are datetime fields, and represent the earliest permitted creation date, and the latest permitted creation date respectively.  You may also optionally filter on only certain types of activities, using the activityTypeIds filter.  This is very useful for removing results which are not relevant for your use case.

The job now has a status of “Created,” but it is not yet in the processing queue.  To put it in the queue so it can begin processing, we need to call the Enqueue Export Activity Job endpoint using the exportId from the creation status response.

Now the status is reporting that the job has been queued.  When a worker becomes available for this job, then the status will be switched to “Processing” and the job will begin aggregating records from Marketo.

Polling Job Status

Note: Job status can only be retrieved for jobs created by the same API user.

Marketo’s Bulk Activity Extract is an asynchronous endpoint, so the job status must be polled to determine when the job is complete.  Poll using the Get Export Activity Job Status endpoint as follows:

The status field may respond with any one of:

  • Created
  • Queued
  • Processing
  • Cancelled
  • Completed
  • Failed

Retrieving Your Data

Once the job is complete, you can retrieve your data using the Get Export Activity File endpoint.

The response will contain a file formatted in the way that the job was configured.  The endpoint will respond with the contents of the file.

If a requested lead field is empty (contains no data), then null will be placed in the corresponding field in the export file.  In the example below, the campaignId field for the returned activity is empty.

To support partial and resumption-friendly retrieval of extracted data, the file endpoint optionally supports the HTTP header Range of the type bytes.  If the header is not set, the whole of the contents will be returned.  You can read more about using the Range header with Marketo Bulk Extract files here.

Cancelling a Job

If a job was configured incorrectly, or becomes unnecessary, it can be easily cancelled using the Cancel Export Activity Job endpoint:

This will respond with a status indicating that the job has been cancelled.