GraphQL
One document per resource, POSTed as `{query, operationName, variables}`.
errors[] is an error, even with HTTP 200
If the response carries a non-empty errors[], the work item fails. That is the default and it is the
right default: partial data silently loaded is worse than a failed run. Set partial_data: accept to
yield the rows anyway and record the error as a warning event.
Pagination and cost
relay pagination reads pageInfo.hasNextPage / endCursor and re-issues with after.
offset_variables covers offset-style schemas. When the schema exposes a query cost, cost_path feeds
the rate limiter directly, so Ingest slows down before the API starts rejecting it.
Data path
$.data.<field>.edges[*].node is supported, where [*] expands arrays and .node unwraps. Plain
lists work too. Introspection is an authoring-time tool only; it never runs during a pipeline.
Credential proof
Introspection, or a trivial query.
Questions
- Which GraphQL sources has Ingest certified?
- Shopify. Each one has run against the real API, with exact row counts asserted.
- Can Ingest sync any GraphQL API?
- Ingest authors the connector from the vendor's published API documentation, runs it against the real API, and lists it as deployable only once it passes. A connector is a declarative spec, never per-source code, which is why a new one is authored rather than programmed.
- Where can GraphQL data land?
- Amazon S3, Google Cloud Storage, Azure Blob Storage, Apache Iceberg, Ingest Managed Lakehouse, MotherDuck, Postgres, Amazon Athena, Databricks, Redshift, BigQuery, ClickHouse, MySQL / generic SQL and Snowflake. The data lands in a destination you own, under credentials you control. Ingest keeps no copy of it.
GraphQL sources, and where they land
Certified GRAPHQL connectors: Shopify.
The other source kinds Ingest reads: File, gRPC, REST, SOAP, SQL, WebSocket.
Where the tables land: Amazon S3, Google Cloud Storage, Azure Blob Storage, Apache Iceberg, Ingest Managed Lakehouse, MotherDuck, Postgres, Amazon Athena, Databricks, Redshift, BigQuery, ClickHouse, MySQL / generic SQL, Snowflake.