NOTE: This is a guest blog post. Ed Blachman is a Senior Architect at TIBCO Software, a well-known vendor of enterprise software. Ed is working on products that allow what Gartner calls “citizen developers” to integrate the cloud services they use without needing to do any programming themselves.
Marketo’s SOAP API is a powerful tool by which developers can harness the power of Marketo and integrate it with our own applications. Between the formal documentation and the community resources, there’s a lot of information available regarding how to use it. When I was getting started, I leaned heavily on that information and found it invaluable. However, in that process, I built up some tips and tricks that I hadn’t seen in any of those places. Here’s some of what I figured out.
The Developers’ Sandbox
The Sandbox is, of course, a wonderful resource for API developers: a safe place in which you can experiment with Marketo features, adding and removing objects without interfering with real marketing activities carried on by your organization’s actual Marketo users. However, the Sandbox is not a panacea. For example, I needed to share our Sandbox with another development group, and this took some doing, because they had gotten accustomed to the notion that they owned the Sandbox. Eventually, we figured out a couple of Best Practices for sharing:
– Don’t write tests that depend on complete knowledge of the contents of your Sandbox. As a shared resource, schemas may be subject to change without notice, as well as entire entries in your leads database or programs or other entities. If your tests assume complete knowledge of the Sandbox, your development cycle will create blackout periods for the groups with whom you’re sharing it. Since typically their development cycle will not coincide with yours, this amounts to hogging the resource–not cool. It’s also not necessary, if you think it through.
– Do use a convention to label all of your stuff–your leads, your lead schema fields, your programs, whatever. If you each can identify your own objects, and if you can agree with your co-tenants that each of you will leave the others’ objects alone, you should be on a firm foundation for sharing. For leads, you could create a custom field, and create a convention using this custom field to identify these leads as your test leads. For lists or programs, you might start the names of your objects with some string that identifies those objects as belonging to you.
– Consider writing tests that clean up after themselves–that first create the objects you’re interested in, then access or update or selectively delete them, then finally remove them. (Note that this is not achievable 100% in the SOAP API because not everything in the Sandbox, or in a real instance for that matter, can be managed via the SOAP API. Even so, it’s still worthwhile to do this as much as you can.)
Real Instances
The problem with the Sandbox is that it’s not being used in production, so it’s difficult to get a sense of what real usage looks like in a Marketo instance. Now, if you’re lucky enough to have a Marketo power user on your team, or if you’re doing bespoke development for internal Marketo users, that’s not such a problem. But in the case of my team, it was a big deal indeed. None of us were Marketo experts, and since we were being asked to understand a large number of cloud services, we just didn’t have the headcount to become experts in anything. Here are some of the insights we gleaned from access to a real instance:
– Large lead schemas. The lead schema in the production instance we accessed has over 200 fields. That made it crystal clear to our UI designers that the UI they were designing had to accommodate schemas of that size (or larger).
– Bursty usage. We saw two orders of magnitude difference between the highest-usage times and low-usage times (in terms of numbers of leads created or updated). This impacted both the volume of data we’d get back from API calls (obvious) and the time it would take for an API call to respond (possibly less obvious).
API Call Response Time
Depending on the time of day, the details of your API call, and the contents of your instance, you may find the SOAP API’s response time will take longer than average. On occasion, we had API calls that took a minute and a half to respond. You need to be aware of the possibility in order to deal with it:
– Test. Maybe this is not a problem for your usage. But don’t just assume that, do some testing.
– Tweak your usage. In our case, the biggest issue was that we set the page size for our calls to getMultipleLeads to be as big as the API allows. In our context, that makes a certain amount of sense, because our goal is to be as efficient as possible with our customer’s API quota. But in your context, you may not need to worry so intensely about your users’ API call quotas, in which case you’ll definitely get better response time by asking for smaller pages of data.
Lead Partitioning
Marketo provides powerful tools–partitions and workspaces–that allow multiple marketing groups to share a single Marketo instance. However, those tools aren’t reflected directly in the SOAP API. For instance, when you use getMultipleLeads to get all leads that have been updated or created since some datetime, you get back all the leads in your instance for which that’s the case, without regard for (and with nothing to indicate) which partition or workspace contains any given lead. Lead creation and adding leads to lists are other contexts in which lead partitioning may impact what your API calls actually do. Note that this means that partitions and workspaces may not be the solution you’ll need to the problem of Sandbox sharing discussed above.
So how do you figure out whether this is an issue for you? I’ve found all of these to be helpful:
– Marketo Developer Support. The Developer Evangelists are committed to our success in using the APIs, and where there are questions, they’re amazingly good at working to find answers.
– API Documentation. The Evangelists have already brought this issue into some of the documentation, and as part of their commitment to our success, they’re really good about updating the doc.
– Your Own Test Cases. Although using partitions and workspaces for sharing the sandbox may not be a great idea, the Sandbox is a great place to play with partitions and workspaces to figure out whether they pose challenges for your intended usage. (It’s also a good way to narrow down your questions for the Evangelists, which is always a good idea.)
TIMTOWTDI and Testing
“There is more than one way to do it”–the Perl programming motto–actually applies in certain contexts to the Marketo SOAP API. For instance, I wanted to combine updating a set of leads with adding those leads to some list. The SOAP API gives you two ways to do this:
1. importToList + getImportToListStatus. Reading the documentation, this is obviously the “normal” way to do this. However, the fact that you have to poll for the status of your import operation raised a yellow flag for me. Was this really the way I wanted to implement my import?
2. syncMultipleLeads + listOperation. This seems much less elegant than a unitary importToList call, but it doesn’t rely on polling. Was it a viable option?
Cases like these are hard for the Evangelists to deal with, because they really depend on the nature of the instances that you’re dealing with and exactly what you’re trying to do. Luckily, if you’ve set up a robust unit testing environment, you should be able to use it to explore questions like these as well. In this particular case, it turned out that option 2 was better for my use case than option 1–not because of the polling but rather because I ran into field-oriented limitations on importToList, and also because I was trying to write code that could be used in contexts and instances over which I had no control. But your use case may be different–and testing is the only way you’ll find out.
Conclusion
I don’t think any of this is a huge secret. On the other hand, I’d have been ahead of the game if I’d known all this before I got started. I hope you find it useful.