setup

Airbyte

Purpose

Ingests Airbyte organizations, workspaces, users, sources, destinations, tags, and connections into Subimage to map your data-pipeline footprint and its integrations with external systems.

tip

Secret fields below accept either an AWS Secrets Manager ARN or a value pasted directly into Subimage's managed vault. See Secrets for details.

Required Fields

Field Secret? Description
airbyte_client_id No Airbyte application client ID
airbyte_client_secret Yes Airbyte application client secret
airbyte_api_url No Airbyte API base URL (default https://api.airbyte.com/v1, override for self-hosted)

Required Permissions

Subimage syncs every organization the application's user belongs to. What it can collect depends on that user's roles:

Data Role needed in each organization
Organizations, workspaces, sources, destinations, tags, connections Read access to the workspaces, for example Workspace Reader
Users and their organization and workspace permissions Organization Admin

Airbyte only lets an Organization Admin read other users' permissions. If the user is not an Organization Admin of an organization, Subimage still syncs that organization's workspaces and pipelines, and keeps the users from the last complete sync.

Setup Steps

  1. In the Airbyte admin panel, open Settings → Applications and create a new application. The application has the permissions of the user who creates it, so create it as a user with the roles in Required Permissions. Copy the Client ID and Client Secret.
  2. In Subimage, fill in:
    • airbyte_client_id — the Client ID
    • airbyte_client_secret — the Client Secret (or its AWS Secret ARN)
    • airbyte_api_url — leave empty for Airbyte Cloud; set your base URL (e.g. https://airbyte.internal/api/public/v1) for self-hosted
  3. Save the module.

Subimage exchanges the client credentials for a short-lived access token at sync time.

Notes

  • If the application's owning user loses access to a workspace, that workspace drops from the sync. Use a stable service account.
  • To refresh users and permissions for such an organization, give the application's owning user the Organization Admin role there; the next sync picks it up.