Blueprint: External Automation Integrations (Outbound)

Purpose

Define a provider-agnostic architecture for outbound automation integrations (Zapier, Make, Pipedream, custom webhooks) that is easy to extend with new providers and event types without coupling to vendor-specific logic.

Scope

• Outbound event delivery (webhook-style). No polling in the initial design.
• Tenant-scoped subscriptions and API key authentication.
• Provider adapters that enforce URL policy and request formatting.
• Delivery pipeline with retries and observability.

Principles

• Provider-agnostic: providers are adapters, not architectural forks.
• Stable contracts: event envelope and schemas are versioned and long-lived.
• Safe by default: SSRF controls, rate limits, and PII minimization.
• Async delivery: user-facing mutations should not block on outbound calls.
• Observable: delivery success/failure is measurable and debuggable.

Core Concepts

Provider

A named integration vendor or mechanism.

• Examples: zapier, make, pipedream, custom-webhook
• Responsibilities:
  • URL policy (allowed hostnames, required URL patterns, HTTPS enforcement)
  • Auth behavior (API key verification, signing headers)
  • Delivery formatting (headers, payload transforms if needed)
  • Response handling: interpreting the immediate HTTP response; for providers that return a request/task ID (e.g. Zapier) instead of immediate success/failure, optional async status verification (poll or callback) to determine final delivery outcome.

Event Type (Domain Event)

A stable, namespaced identifier describing what happened in the domain.

• Examples:
  • customer.created
  • reservation.created
  • reservation.status_changed
  • reservation.completed
• Guidance:
  • Prefer domain events over vendor trigger names.
  • Start with a small set of high-value, well-defined events.

Subscription

Represents “this tenant wants to receive event X via provider Y at target Z”.

• Suggested fields:
  • tenantId
  • provider (string, e.g. zapier)
  • eventType (string, e.g. reservation.created)
  • targetUrl (validated per provider policy)
  • isActive
  • config (JSON for provider-specific fields, optional)
  • timestamps + optional revokedAt / disabledReason
• Multiple subscriptions per eventType are allowed (e.g. different target URLs).

---

Event Envelope (Stable Contract)

All outbound events use a common envelope so consumers get consistent metadata and schemas can evolve safely.

{
  "id": "evt_01J... (unique, stable)",
  "type": "reservation.status_changed",
  "version": 1,
  "occurredAt": "2026-01-08T12:34:56.789Z",
  "tenantId": "tenant_123",
  "data": { "reservationId": "ord_123", "from": "pending", "to": "confirmed" }
}

Guidance:

• Idempotency: id must be globally unique and stable. Adapters can pass it through as an idempotency key header.
• Versioning: bump version when data changes in a backward-incompatible way.
• PII minimization: prefer IDs + stable references over raw PII unless strictly required.

Event Payload Schemas

interface BookingCreatedEvent {
  id: string;
  type: 'reservation.created';
  version: 1;
  occurredAt: string;
  tenantId: string;
  data: {
    reservationId: string;
    customerId: string;
    status: string;
    rentalStartDate: string;
    rentalEndDate: string;
    totalAmount: number;
    currency: string;
    items: Array<{ inventoryId: string; name: string; quantity: number }>;
  };
}

interface BookingStatusChangedEvent {
  id: string;
  type: 'reservation.status_changed';
  version: 1;
  occurredAt: string;
  tenantId: string;
  data: {
    reservationId: string;
    customerId: string;
    from: string;
    to: string;
    reason?: string;
  };
}

interface CustomerCreatedEvent {
  id: string;
  type: 'customer.created';
  version: 1;
  occurredAt: string;
  tenantId: string;
  data: {
    customerId: string;
    email: string;
    name: string;
    phone?: string;
    companyName?: string;
  };
}

---

Provider Adapter Interface

interface UrlPolicy {
  allowedHostnames: string[] | null; // null = tenant-controlled (strict SSRF)
  enforceHttps: boolean;
  allowRedirects: boolean;
}

interface ProviderAdapter {
  readonly provider: string;
  readonly urlPolicy: UrlPolicy;

  validateUrl(url: string): { ok: true } | { ok: false; reason: string };

  verifyApiKey(keyPlaintext: string): Promise<{ tenantId: string } | { error: string }>;

  buildRequest(subscription: { targetUrl: string; config?: unknown }, envelope: unknown): {
    url: string;
    method: 'POST';
    headers: Record<string, string>;
    body: string;
    timeoutMs: number;
  };
}

Each provider (Zapier, Make, custom-webhook) implements this interface with its own URL allowlist and signing logic.

Async response / status verification (optional)

Some providers (e.g. Zapier) respond to the webhook with 200/202 and a request or task ID rather than an immediate success/failure. Final delivery outcome is only known after checking the provider’s status API (or a callback) later.

• Recommendation: Treat the initial send as “accepted” and record the provider’s request/task ID in integration_delivery_attempts (e.g. providerRequestId). A separate background job or provider-specific worker can optionally poll the provider’s status endpoint and update the attempt’s final status (e.g. delivered vs failed).
• Provider adapter: An optional method such as getDeliveryStatus?(providerRequestId: string, subscription): Promise<'pending' | 'delivered' | 'failed'> can be added to the adapter interface when this capability is needed; providers that don’t support it omit the method.
• MVP: The blueprint does not require async verification for launch. Immediate response (status code, body) is enough for retries and basic observability; async verification can be added per provider when needed.

---

Delivery Pipeline (Outbound)

Delivery should be a pipeline, not a one-off fetch() from business logic.

Suggested flow:

Domain action (service / mutation)
  -> emit IntegrationEvent(type, version, data)
  -> IntegrationDispatcher
       -> load active subscriptions for (tenantId, eventType)
       -> ProviderAdapter.validateUrl(targetUrl)
       -> ProviderAdapter.buildRequest(subscription, eventEnvelope)
       -> send (with timeout)
       -> record attempt + metrics
       -> retry/backoff when applicable

Notes:

• Async-by-default: delivery should not block user-facing mutations. Use background jobs when possible.
• Retry policy:
  • retry on network errors / 5xx
  • no retry on 4xx (except 429 with Retry-After, if supported)
  • exponential backoff with a small max attempt count for MVP
• Queue optionality: a queue/workflow layer (e.g. QStash / Upstash workflow) can be introduced without changing the event/subscription model.

---

Operational Defaults

Retry Policy

• Max attempts: 3
• Backoff: exponential (1s → 2s → 4s)
• Retry on: network errors, 5xx, 429 (respect Retry-After)
• No retry: 4xx (except 429), validation failures

Timeouts

• Connection: 5s
• Response: 10s
• Total operation: 15s

Rate Limits (per tenant)

• Subscription create/delete: 60/min
• Auth validation: 120/min
• Outbound delivery: 100 events/min

Payload Constraints

• Max size: 1 MB
• Max URL length: 2048 chars
• Max nesting depth: 5 levels

---

Data Model (Suggested, Extendable)

Avoid a provider-specific schema. Prefer provider-agnostic tables with a provider discriminator.

integration_api_keys

• Purpose: authenticate external callers (Zapier CLI/app, partner tooling) for subscription management.
• Suggested fields:
  • tenantId, provider
  • keyHash (never store plaintext), keyPrefix (for identification)
  • name, scopes (optional), revokedAt, lastUsedAt, timestamps

integration_subscriptions

• Purpose: store subscriptions created via REST Hook registration.
• Suggested fields:
  • tenantId, provider, eventType, targetUrl
  • hookId (string, optional; provider-returned hook ID for unsubscribe—persist from subscribe response; do not rely only on config)
  • isActive, timestamps
  • config (JSON, optional; provider-specific parameters live here)
  • unique constraint (tenantId, provider, eventType, targetUrl)
  • index on hookId for DELETE-by-hook-id lookups

integration_events

• Purpose: store domain events (e.g. transactional outbox) before delivery; referenced by delivery attempts.
• Suggested fields:
  • id (primary key)
  • type (string; e.g. reservation.created, reservation.status_changed, customer.created)
  • version (int; envelope version)
  • occurredAt, tenantId, timestamps
  • data (JSON; event payload per type)
  • index on (tenantId, type) and occurredAt for dispatch and observability

integration_delivery_attempts (optional for MVP)

• Purpose: observability + troubleshooting (debug “why didn’t this fire?”).
• Suggested fields:
  • subscriptionId, eventId, status, statusCode
  • providerRequestId (optional; for providers that return a request/task ID for async status checks)
  • errorCategory, errorMessage (no secrets), timestamps

---

API Surface (Provider-Agnostic)

Keep routes and semantics consistent across providers.

• GET /api/integrations/:provider/auth/test

  • Validates API key; returns { tenantId, provider }.

• POST /api/integrations/:provider/subscriptions (REST Hooks subscribe)

  • Body: { eventType: string, targetUrl: string, config?: object }
  • Returns: { subscriptionId: string, hookId?: string }. Store the provider-returned hook ID in the subscription’s hookId field; include it in the response when the provider returns one.

• DELETE /api/integrations/:provider/subscriptions (REST Hooks unsubscribe)

  • Query: ?subscriptionId={tenant-sub-id} (preferred) or ?providerHookId={provider-hook-id} (or ?hookId=...). When a hook ID is provided, the server MUST query by the subscription table’s hookId field (e.g. WHERE hookId = ?) to resolve the subscription and perform unsubscribe.
  • Or body: { eventType: string, targetUrl: string } (exact unique match).
  • Returns: { deleted: true }.

Optional (later):

• GET /api/integrations/:provider/subscriptions
  • List active for key/tenant; includes hookId / providerHookId when present.

---

Error Handling

Error Response Shape

{
  "error": {
    "code": "invalid_subscription",
    "message": "Target URL must use HTTPS",
    "details": { "field": "targetUrl" }
  }
}

HTTP Status Codes

Code	Meaning	Retry?
401	Invalid/missing API key	No
403	Key revoked or insufficient scope	No
404	Unknown provider or subscription	No
422	Validation failed (SSRF block, bad eventType)	No
429	Rate limited	Yes
500	Server error	Yes

Delivery Failure Categories

For integration_delivery_attempts.errorCategory:

• network_error - timeout, DNS, connection refused
• client_error - 4xx from target
• server_error - 5xx from target
• rate_limited - 429 from target
• ssrf_blocked - URL failed policy

---

Security & Compliance

• API keys: store hashed; rotate and revoke; never log keys.
• SSRF:
  • enforce https:
  • block private IP ranges and link-local targets
  • validate hostnames per provider (allowlist) and disallow redirects across hosts
• Rate limiting: subscription endpoints should be rate limited per tenant/key.
• Timeouts and payload size: hard timeouts and max payload size to prevent resource exhaustion.
• No-throw pattern: when implemented, prefer the repository’s no-throw error handling patterns (and ensure Sentry captures).

---

Observability

• Emit structured logs/metrics for:
  • subscriptions created/removed
  • delivery attempts, retries, final failures
• Capture failures in Sentry with:
  • provider, eventType, subscriptionId, tenantId (non-PII)
  • redacted target URL (hostname only) when needed

---

Extension Points

• Add a provider: implement a new ProviderAdapter and expose routes under /api/integrations/:provider/*.
• Add an event type: define/validate event payload schema, assign a version, and emit it from the correct domain boundary.
• Add reliability features: introduce a queue, delivery attempts table, replay tooling, or batching without changing event envelope semantics.

---

Provider-Specific Plans

Zapier is the first adapter implementing this blueprint. See docs/agent/zapier-integration-plan.md for Zapier-specific scope and MVP sequencing.