Data Integration 101
•
6 MIN READ
So how do you know what the best option is going forward?
This question does get asked on occasion, and while both have their advantages and disadvantages, the process you choose is dependent on the platforms involved. If your ERP lacks APIs or web services, your options become limited to what is possible. Thus, it is important to always consider your integration strategy before purchasing a platform. Does it have the APIs you’ll need? Are the APIs built in a usable and effective way?
Depending on the capability of the platforms involved, you’ll be required to strategize around the limitations it creates. Too often do we see companies make snap decisions on fancy pitches, only to find out they can’t model their operational processes into the platform they’ve paid for. As you can imagine, these decisions can have a massive and negative effect if not properly considered.
We once were asked to vet a Medical Patient compliance system as a potential customer. They wanted to create a unique experience that set them apart from their competition, but due to the regulations around HIPAA, they needed to create a unique and interesting integration strategy. When we sat down with the HIPAA platform, it turned out that there were no APIs or connectivity opinions on the platform at all. Without proper vetting, a poor purchase would have been made, acting in complete counter to the reason the investment was being made.
Take your time, understand what you want to achieve in the big picture, and vet against the operational needs of that picture. Anything less puts your decision on investment at risk.
That being said, some Merchants are stuck with old or legacy platforms, like ERPs, that have been within the business long before APIs were the main form of communication method used.
At eHouse, we see an AS400 (IBM I-Series) project almost weekly. The majority of legacy systems, will have an import-export process. And while it wasn’t originally designed for cloud-based applications, the import/export processes can be modeled in a way that allows it to interact with many cloud applications. Thus your decision rarely comes down to which is better, but rather, you need to take a holistic look at the applications and platforms involved, understand the operations, understand the integration capabilities, and then model a strategy around those systems.
With all of that in mind, it is still interesting and in a lot of cases, important to understand the advantages and disadvantages of both Flat File integrations and API integrations.
To put it simply, Flat File integrations via an sFTP will always be slower than an API integration. In the flat file case, the slower speeds can be mitigated with listening triggers in the sFTP, sFTP integrations will always have a natural chokepoint that will slow down the integration.
Flat File integrations via an sFTP will always be slower than an API integration.
The easiest way to imagine a flat-file integration would be two companies exchanging letters via a secure mailbox. When the integration platform has data for a legacy ERP, for example, the integration platform will drop off a ‘letter’ into the agreed-upon mailbox. At this point, the ERP can take a look into that mailbox, retrieve any of its mail, and process that letter into its system.
Anything coming back the other way, works in the exact same way, dropping off a letter for the integration platform to process. As these systems are interacting via a communication method that lives separate of both systems, it does mean there will be a delay in how the information is processed, as it isn’t going directly into the target system.
APIs are a communication method, with validation built into the API itself.
In contracts, APIs are a communication method, with validation built into the API itself. It's an access point at which data can be pushed directly into or pulled directly from. Due to the direct nature of this process, you naturally speed up the communications between the platforms. You’ll also receive errors directly, allowing for better reconciliation processes and keener automation that allow for reruns of a data payload, without having to disrupt the operations of your day.
Flat-Files Integrations use an FTP of some kind, in order to provide data to and from the cloud-based and Legacy system. As the FTP lives separately from the target systems and acts more like a mailbox, the file standards need to be agreed upon by the developers of the ERP and the Integration company.
Advantages and disadvantages of Flat-File Integrations:
While standards can be pre-created to base the discussion on, the reality is all businesses sell differently, and as a result, require different data sets to see out their operational processes.
Thus a small amount of additional work is required to properly agree and vet a flat file for its intended purpose (ie. Sales Order creation in the ERP).
APIs file framework live within the API itself. The file standards are not agreed upon by the Merchant and Integrator, but given by the developers of the systems involved (ie. Shopify's APIs). This can have its issues though, as no two APIs are built the same and poorly thought out APIs can limit the functionality integrations can provide.
Advantages and disadvantages of APIs:
It’s important to vet the APIs against the functionality you’d like — just because they have APIs does not mean that the platform can provide the integration functionality you need for your operational processes.
Have an idea for a project? Fill out this form and we’ll get back to you shortly.
