System Integration Approaches: Point-to-Point, Middleware, iPaaS or Events?

System integration connects separate applications, such as an ERP, an online store and a CRM, so data moves between them without re-keying. The main approaches are point-to-point APIs, middleware or an enterprise service bus (ESB), integration platform as a service (iPaaS), event-driven messaging and scheduled file transfer. The right choice depends on how many systems you connect, how fast data must move and who runs it.

The pattern matters less than many comparisons suggest. Integrations usually fail for reasons that apply to every pattern: nobody decided which system owns a customer record, a retry created a duplicate order, or errors piled up in a log nobody read. This guide compares the approaches, then covers those decisions in turn.

What are the main system integration approaches?

There are five common approaches, and most organisations end up using more than one.

• Point-to-point APIs. Each system calls the other directly over its API. Simple to start; the number of connections grows quickly as systems are added.
• Middleware or ESB. A central layer receives messages, transforms and routes them. Systems connect to the hub, not to each other. Usually self-hosted and run by an in-house team.
• iPaaS. A cloud-hosted integration platform with prebuilt connectors, visual mapping and managed hosting. Similar idea to middleware, run as a subscription service.
• Event-driven. Systems publish events ("order placed", "payment captured") to a message broker or event stream; interested systems subscribe. Publishers do not need to know who is listening.
• File-based. Systems exchange files (CSV, XML, fixed-width) on a schedule, typically over SFTP. Old, unglamorous and still the right answer in plenty of cases.

How do the integration approaches compare?

Each approach trades simplicity against central control and operating effort.

Which system should be the system of record?

Before choosing a pattern, decide which system owns each data object: the one place where it is created and corrected. Every other system receives a copy and does not edit it, or sends changes back through a defined route.

An illustrative ownership table for a business running ERP, e-commerce, CRM and online payments:

Two systems both editing the same field, with "last write wins", is the most common source of data that quietly drifts apart.

Should integrations be synchronous or asynchronous?

Use synchronous calls when the user is waiting and needs the answer now; use asynchronous messaging when the work can complete a moment later. Most business integrations should be asynchronous.

• Synchronous: checking stock or a customer's credit before confirming an order; validating an address. The caller waits, so the other system's downtime becomes your downtime. Set timeouts and decide what happens when they expire.
• Asynchronous: sending orders to the ERP, updating the CRM after an invoice, syncing product changes. A queue absorbs bursts and outages, and work catches up when the other system returns.

How do idempotency and retries prevent duplicates?

Networks fail mid-request, so integrations retry. Idempotency makes a retry safe: sending the same request twice has the same effect as sending it once.

RFC 9110, the HTTP standard, defines GET, PUT and DELETE among the idempotent methods; POST, the usual way to create records, is not. So creation needs a design choice. Common techniques:

• Idempotency keys. The caller sends a unique key with each create request and reuses it on retry. Stripe's API documents this pattern: requests carrying the same Idempotency-Key header return the original result instead of creating a second object.
• Natural keys. Create the ERP order with the web order number as an external reference, and reject a second order with the same reference.
• Upserts. "Create or update by external ID" rather than "create".
• Processed-message log. Record the ID of each message handled and skip repeats. Stripe's webhook documentation, for example, warns that an endpoint may receive the same event more than once and that delivery order is not guaranteed, and recommends logging processed event IDs.

Retries need limits. Retry transient failures (timeouts, temporary unavailability) with increasing delays plus a random element, so many clients do not retry in lockstep; do not retry errors that will fail again, such as validation errors. When an API returns 429 Too Many Requests, defined in RFC 6585, it may include a Retry-After header saying how long to wait; honour it.

What happens to messages that fail?

A failed message should land in an error queue, often called a dead-letter queue, with the reason, the payload and a way to fix and replay it. It should never vanish into a log file.

Give the error queue an owner in the business, not only in IT, because many errors are data problems: an unknown product code, a customer without a tax number. Then add reconciliation reports, which catch what error handling misses:

• Orders in the store versus orders in the ERP, by day
• Payments captured by the provider versus payments recorded in the ERP
• Stock levels in the ERP versus the store, by item
• Customer counts and recent changes in the CRM versus the ERP

Reconciliation is what tells you an integration is correct rather than merely running. The same principle applies to data migration; see our ERP data migration plan.

How should integrations be monitored?

Monitor business outcomes as well as technical health. An integration can be "up" while silently processing nothing.

Integration monitoring checklist:

• Message throughput per flow, with alerts when it drops to zero during business hours
• Error-queue depth and age of the oldest unresolved error
• Latency from source event to target update
• Failed authentication and permission errors, which often signal expired credentials
• Rate-limit responses from vendor APIs
• A correlation ID carried through every hop, so one order can be traced across systems
• A daily reconciliation summary sent to the business owner

How should integration security be handled?

Each integration should run under its own service account with only the permissions it needs, credentials kept in a secrets store and rotated, and traffic encrypted.

• Least privilege. NIST SP 800-53 (Rev. 5) control AC-6 calls for allowing only the access needed for assigned tasks, and applies it to processes acting for users as well as people. An order-import integration should create orders, not read payroll.
• One account per integration. Shared credentials make it impossible to tell which integration did what, or to revoke one without breaking others.
• Secrets management. No credentials in code, scripts or spreadsheets. Rotate on a schedule and when people leave.
• Verify inbound calls. Check webhook signatures where the sender provides them; Stripe, for instance, signs each webhook event and documents how to verify it.
• Encrypt in transit, including file transfers: use SFTP rather than plain FTP.
• Log access without logging sensitive payload fields.

How do vendor API limits affect integration design?

Every vendor API you depend on has rules you do not control: rate limits, page sizes, which events it can notify you about, how long it retries notifications, and when old API versions are retired. Treat them as dependencies and check them in the vendor's current documentation before design is fixed.

Questions to answer for each vendor:

• What are the documented rate limits, and do they apply per account, per user or per app?
• Does the API support webhooks for the events you need, or must you poll?
• Are webhooks retried, for how long, and can they arrive more than once or out of order?
• Is there a bulk or batch endpoint for large syncs?
• How are API versions announced and retired?
• Is there a sandbox that behaves like production?

Design for the answers: queue and throttle outbound calls, use bulk endpoints for initial loads, and run a scheduled catch-up job in case notifications are missed.

Is file-based integration still a valid choice?

Yes. Scheduled files over SFTP remain a sound choice where data is naturally batched (daily bank statements, payroll exports, nightly price lists), where a partner or legacy system has no usable API, or where an auditable file per run is useful.

Make files reliable with a few rules: a fixed, documented layout with a header row and version; a control record or trailer with row count and totals; write to a temporary name and rename when complete, so partial files are never picked up; unique file names with dates; archive every processed file; and reject a file whose name has already been processed. If a legacy system can only produce files, that is often the first step in a legacy software modernization plan rather than a reason to replace it immediately; our legacy system modernization options guide compares the routes.

Decision checklist: which integration approach fits?

1. How many systems and flows? Two or three systems with a handful of flows: point-to-point is usually fine. Growing beyond that: consider a hub, iPaaS or events.
2. Where do the systems live? Mostly SaaS with available connectors: iPaaS is worth evaluating. Mostly on-premise or custom: middleware or custom integration services.
3. How fast must data move? Seconds: APIs or events. Hours: scheduled jobs or files.
4. How many consumers per event? Several systems reacting to the same event points to event-driven design.
5. Who will run it? No integration team: favour managed services and fewer moving parts. An in-house platform team: a self-hosted hub becomes realistic.
6. What do vendor limits allow? Rate limits and webhook support may decide the pattern for you.
7. Can you reconcile it? Whatever you choose, plan the reconciliation report first.

Illustrative scenario: ERP, e-commerce, CRM and payments

The following scenario is illustrative; the business and its systems are generic.

A distributor runs an ERP, an online store, a CRM and an online payment provider. The first design connected everything point to point, with the store and the CRM both editing customer addresses. Duplicate orders appeared whenever the ERP was slow and the store retried.

The revised design:

1. Ownership agreed as in the system-of-record table above.
2. Payments: the provider's signed webhook updates payment status. The handler logs event IDs and ignores repeats, and the order is marked paid only by that notification, never by the customer's browser returning.
3. Orders: the store publishes "order placed" to a queue. An integration service creates the ERP order using the web order number as an external reference, so a retry cannot create a second order. Failures go to an error queue owned by the sales operations lead.
4. Stock and prices: ERP changes are pushed to the store, throttled to stay within the store platform's documented rate limits, with a nightly full refresh as a safety net.
5. CRM: receives customer and invoice summaries from the ERP asynchronously; marketing preferences flow from the CRM to the store.
6. Bank statements: a daily file over SFTP to the ERP for payment matching.
7. Reconciliation: a morning report compares orders, payments and stock across systems and flags differences.

Four approaches in one estate, each chosen for the flow it serves.

When is building an integration the wrong move?

Sometimes the better answer is not to integrate. If two systems overlap heavily, consolidating onto one may remove the need for a fragile sync. If a flow happens a few times a month, a documented manual export may be cheaper and safer than an integration nobody monitors. And if the data in either system is unreliable, integration spreads the problem faster; clean it first.

How Timeline Digital approaches integration

We start with the ownership table and the reconciliation reports, then choose the pattern per flow. Our systems and API integration page describes the work, ERP integration services covers connecting ERP platforms, and enterprise system integrations explains larger multi-system programmes. Timeline Digital starts with a free pilot of 2 to 3 key modules before the full project, so you can see the most critical flows working, with monitoring and reconciliation, before the rest is scoped.