Bulk Lead Extract

Bulk Lead Extract Endpoint Reference

The Bulk Lead Extract set of REST APIs provides a programmatic interface for retrieving large sets of lead/person records out of Marketo.  Additionally, it can be used to retrieve leads incrementally based on the created date of the record, the most recent update, static list membership, or smart list membership.  This the recommended interface for use cases which require continuous interchange of data between Marketo and one or more external systems, for ETL, data warehousing, and archival purposes.

Permissions

The Bulk Lead Extract APIs require that the owning API user have a role with one or both of the Read-Only Lead, or Read-Write Lead permissions.

Filters

Leads support a variety of filter options.  Certain filters, including the updatedAt, smartListName, and smartListId require additional infrastructure components which have not yet been rolled out to all subscriptions.  Only one filter type may be specified per export job.

Filter Type Data Type Notes
createdAt Date Range Accepts a JSON object with the members startAt and endAt.  startAt accepts a datetime representing the low-watermark, and endAt accepts a datetime representing the high-watermark.  The range must be 31 days or fewer.  Datetimes should be in an ISO-8601 format, without milliseconds.

Jobs with this filter type will return all accessible records which were created within the date range.

updatedAt* Date Range Accepts a JSON object with the members startAt and endAt. startAt accepts a datetime representing the low-watermark, and endAt accepts a datetime representing the high-watermark. The range must be 31 days or fewer.  Datetimes should be in an ISO-8601 format, without milliseconds.

Note: This filter does not filter on the visible “updatedAt” field which only reflects updates to standard fields.  It filters based on when the most recent field update was made to a lead record

Jobs with this filter type will return all accessible records which were most recently updated within the date range.

staticListName String Accepts the name of a static list.  Jobs with this filter type will return all accessible records which are members of the static list at the time that the job begins processing.

Retrieve static list names using the Get Lists endpoint.

staticListId Integer Accepts the id of a static list. Jobs with this filter type will return all accessible records which are members of the static list at the time that the job begins processing.

Retrieve static list ids using the Get Lists endpoint.

smartListName* String Accepts the name of a smart list.  Jobs with this filter type will return all accessible records which are members of the smart lists at the time that the job begins processing.

Retrieve smart list names using the Get Smart Lists endpoint.

smartListId* Integer Accepts the id of a smart list.  Jobs with this filter type will return all accessible records which are members of the smart lists at the time that the job begins processing.

Retrieve smart list ids using the Get Smart Lists endpoint.

* Filter type is unavailable for some subscriptions.  If unavailable for your subscription, you will receive an error when calling the Create Export Lead Job endpoint (“1035, Unsupported filter type for target subscription”).  Customers may contact Marketo Support to have this functionality enabled in their subscription.

Options

The Create Export Lead Job endpoint provides several formatting options, giving the user the ability to include particular fields within the exported file, the ability to rename column headers of these fields, and the format of the exported file.

Parameter Data Type Required Notes
fields Array[String] Yes The fields parameter accepts a JSON array of strings.  Each string must be the REST API name of a Marketo lead field.  The listed fields will be included in the exported file.  The column header for each field will be the REST API name of each field, unless overridden with columnHeader.

Note: When the Adobe Experience Cloud Audience Sharing feature is enabled, a cookie sync process occurs that associates Adobe Experience Cloud ID (ECID) with Marketo leads.  You can specify the “ecids” field to include ECIDs in the export file.

columnHeaderNames Object  No A JSON object containing key-value pairs of field and column header names.  The key must be the name of a field included in the export job.  This is the API name of the field which can be retrieved by calling Describe Lead.  The value will be the name of the exported column header for that field.
 format String No Accepts one of: CSV, TSV, SSV.  The exported file will be rendered as a comma-separated values, tab-separated values, or space-separated values file, respectively if set.  Defaults to CSV if unset.

Creating a Job

The parameters for the job are defined before kicking off the export using the Create Export Lead Job endpoint.  We will need to define the fields that are needed for export, the type of parameters of the filter, the format of the file, and the column header names, if any.

This request will begin export a set of leads created between January 1st 2017, and January 31st 2017, including the values from the corresponding firstName, lastName, id, and email fields.

This returns a status response indicating that the job has been created.  The job has been defined and created, but it hasn’t yet been kicked off.  To do so, the Enqueue Export Lead Job endpoint must be called using the exportId from the creation status response:

This will respond with a status of “Queued” after which it will be set to “Processing” when there is an available export slot.

Polling Job Status

Note: Status can only be retrieved for jobs which were created by the same API user.

Since this is an asynchronous endpoint, after creating the job, we will need to poll its status to determine its progress.  Poll using the Get Export Lead Job Status endpoint.  The status is only updated once every 60 seconds, so a polling frequency lower than this is not advised, and in nearly all cases is still excessive.  Let’s take a quick look at polling.

The status endpoint responds indicating that the job is still processing, so the file is not yet available for retrieval.  Once the job status changes to “Completed” it will be prepared for download.

The status field may respond with any one of:

  • Created
  • Queued
  • Processing
  • Cancelled
  • Completed
  • Failed

Retrieving Your Data

To retrieve the file of a completed lead export, simply call the Get Export Lead File endpoint with your exportId.

The response will contain a file formatted in the way that the job was configured.  The endpoint will respond with the contents of the file.

If a requested lead field is empty (contains no data), then null will be placed in the corresponding field in the export file.  In the example below, the email field for the returned lead is empty.

To support partial and resumption-friendly retrieval of extracted data, the file endpoint optionally supports the HTTP header Range of the type bytes.  If the header is not set, the whole of the contents will be returned.  You can read more about using the Range header with Marketo Bulk Extract files here.

Cancelling a Job

If a job was configured incorrectly, or becomes unnecessary, it can be easily cancelled using the Cancel Export Lead Job endpoint:

This will respond with a status indicating that the job has been cancelled.