Marketo provides interfaces for insertion of large sets of person and person related data, called Bulk Import. Currently, interfaces are offered for three object types:
- Leads (Persons)
- Custom Objects
- Program Members
Bulk import is performed by creating a job, and then waiting for the job to complete reading a file. These jobs are executed asynchronously, and can be polled to retrieve the status of the import. Files are uploaded using HTTP multipart/form-data per RFC 2399.
Note: Bulk API endpoints are not prefixed with ‘/rest’ like other endpoints.
Authentication
The bulk import APIs use the same OAuth 2.0 authentication method as other Marketo REST APIs. This requires a valid access token to be embedded either as the query-string parameter “access_token={AccessToken}”, or as an HTTP header “Authorization: Bearer {AccessToken}”.
Limits
- Max Concurrent Import Jobs: 2
- Max Queued Import Jobs (inclusive of currently importing jobs): 10
- Max Size of Import File: 10 MB
Permissions
Bulk Import uses the same permissions model as the Marketo REST API, and does not require any additional special permissions in order to use, though specific permissions are required for each set of endpoints.
Record Operations
Bulk import is an “insert or update” record operation. If a matching record is found in the database, it is updated. Otherwise, a new record is created. The bulk import response does not indicate whether a given record was updated or inserted.
Creating a Job
Marketo’s bulk import APIs use the concept of a job for executing data import. Let’s look at creating a simple lead import job using the Import Leads endpoint. Note that this endpoint uses multipart/form-data as the content-type. This can be tricky to get right, so best practice is to use an HTTP support library for your language of choice. If you are just getting your feet wet, we suggest that you use curl.
1 |
POST /bulk/v1/leads.json?format=csv |
1 2 3 |
Content-Type: multipart/form-data; boundary=--------------------------WebKitFormBoundaryBQACkJZyaiIAXogC Content-Length: 311 Host: <munchkinId>.mktorest.com |
1 2 3 4 5 6 7 8 9 |
------WebKitFormBoundaryBQACkJZyaiIAXogC Content-Disposition: form-data; name="file"; filename="leads.csv" Content-Type: text/csv firstName,lastName,email Able,Baker,ablebaker@marketo.com Charlie,Dog,charliedog@marketo.com Easy,Fox,easyfox@marketo.com ------WebKitFormBoundaryBQACkJZyaiIAXogC-- |
This request will construct a job that will import values contained in the CSV file named “leads.csv” with the column headers “FirstName”, “LastName”, “Email”, “Company”.
1 2 3 4 5 6 7 8 9 10 11 |
{ "requestId": "d01f#15d672f8560", "result": [ { "batchId": 3404, "importId": "3404", "status": "Queued" } ], "success": true } |
When we submit the job it will return a batchId, which we can then use to check its status.
Common Parameters
Each job creation endpoint shares some common parameters for configuring the file format, field names, and filter of a bulk extract job. Each subtype of extract job may have additional parameters:
Parameter | Data Type | Notes |
format | String | Determines the file format of the imported data with options for comma-separated values, tab-separated values, and semi-colon-separated values. Accepts one of: CSV, SSV, TSV. The format defaults to CSV. |
file | String | Data is specified through multipart form-data in the file. |
Polling Job Status
Determining the status of the job is simple using the Get Import Lead Status endpoint.
1 |
GET /bulk/v1/leads/batch/{batchId}.json |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
{ "requestId": "1f63#15d6738fd15", "result": [ { "batchId": 3404, "importId": "3404", "status": "Complete", "numOfLeadsProcessed": 3, "numOfRowsFailed": 0, "numOfRowsWithWarning": 0, "message": "Import succeeded, 3 records imported (3 members)" } ], "success": true } |
The inner status member will indicate the progress of the job, and may be one of the following values: Queued, Importing, Complete, Failed. In this case our job has completed, so we can stop polling.
Failures
Failures are indicated by the numOfRowsFailed attribute in Get Import Lead Status response. If numOfRowsFailed is greater than zero, then that value indicates the number of failures that occurred.
To retrieve the records and causes of failed rows, you’ll need to retrieve the failure file using the Get Import Lead Failures endpoint.
1 |
GET /bulk/v1/leads/batch/{batchId}/failures.json |
The file indicates which rows failed, along with a message indicating why the record failed.