Ingest documentation
From any API to tables you can query
Ingest authors the connector, runs the pipeline and lands typed, deduplicated tables in a destination you own. These pages cover what it connects, what it writes, and how to set each destination up.
Start here
Data sources
Every source: the connectors certified against a real API today, and the vendors Ingest builds on request.
The tables Ingest creates
What lands in your warehouse (data tables, child tables, quarantine, the three _ingest_ control tables) and the semantics of each.
Machine-readable index
The same contents as plain text, for AI crawlers and agents.
Connectors
abilene_311rest · public_data · beta
albanyny_311rest · public_data · beta
annarbor_311rest · public_data · beta
annearundelcounty_311rest · public_data · beta
anthropicrest · analytics · beta
asanarest · productivity · beta
auburnwa_311rest · public_data · beta
aurorail_311rest · public_data · beta
austintexas_311rest · public_data · beta
baltimore_311rest · public_data · beta
batonrouge_311rest · public_data · beta
beaumont_311rest · public_data · beta
bellevue_311rest · public_data · beta
bellingham_311rest · public_data · beta
berkeley_311rest · public_data · beta
bethlehempa_311rest · public_data · beta
bloomingtonin_311rest · public_data · beta
bocaraton_311rest · public_data · beta
boston_311rest · civic · beta
bridgeport_311rest · public_data · beta
brockton_311rest · public_data · beta
brookline_311rest · public_data · beta
buffalony_311rest · public_data · beta
burlingtonvt_311rest · public_data · beta
cambridge_311rest · public_data · beta
cambridgemarest · public_data · beta
charlestoncountysc_311rest · public_data · beta
charlottenc_311rest · public_data · beta
chulavista_311rest · public_data · beta
cincinnati_311rest · public_data · beta
cityofchicago_311rest · public_data · stable
cityofnewyork_311rest · public_data · stable
claytoncountyga_311rest · public_data · beta
cleveland_311rest · public_data · beta
cobbcountyga_311rest · public_data · beta
columbus_311rest · public_data · beta
corona_311rest · public_data · beta
dallas_311rest · public_data · beta
davenport_311rest · public_data · beta
denver_311rest · public_data · beta
detroit_311rest · public_data · beta
elgin_311rest · public_data · beta
encinitas_311rest · public_data · beta
everettwa_311rest · public_data · beta
federalway_311rest · public_data · beta
fortlauderdale_311rest · public_data · beta
gilbert_311rest · public_data · beta
google_ad_managerrest · advertising · beta
google_analyticsrest · analytics · beta
greenwichct_311rest · public_data · beta
hartford_311rest · public_data · beta
honolulu_311rest · public_data · beta
houston_311rest · public_data · beta
howardcountymd_311rest · public_data · beta
huntingtonwv_311rest · public_data · beta
huntsville_311rest · public_data · beta
indy_311rest · public_data · beta
jerseycity_311rest · public_data · beta
jirarest · productivity · beta
kcmo_311rest · public_data · beta
kentwa_311rest · public_data · beta
knoxcountytn_311rest · public_data · beta
lacity_311rest · public_data · beta
lexingtonky_311rest · public_data · beta
lincolnne_311rest · public_data · beta
linkedin_adsrest · advertising · beta
livonia_311rest · public_data · beta
louisville_311rest · public_data · beta
lubbock_311rest · public_data · beta
maconbibb_311rest · public_data · beta
manchesternh_311rest · public_data · beta
mauicounty_311rest · public_data · beta
medfordma_311rest · public_data · beta
memphis_311rest · public_data · beta
miami_311rest · public_data · beta
miamidade_311rest · public_data · beta
minneapolis_311rest · public_data · beta
montgomerycountymd_311rest · public_data · beta
multnomahcounty_311rest · public_data · beta
naperville_311rest · public_data · beta
nashua_311rest · public_data · beta
nashville_311rest · public_data · beta
newarknj_311rest · public_data · beta
noaarest · weather · beta
nola_311rest · public_data · beta
norfolk_311rest · public_data · beta
oakland_311rest · public_data · beta
omaha_311rest · public_data · beta
openbrewerydbrest · food_and_drink · stable
paterson_311rest · public_data · beta
peoria_311rest · public_data · beta
phila_311rest · public_data · beta
pinellascounty_311rest · public_data · beta
pittsburgh_311rest · public_data · beta
pokeapirest · entertainment · beta
portlandme_311rest · public_data · beta
postgressql · databases · beta
princegeorgescountymd_311rest · public_data · beta
provo_311rest · public_data · beta
raleigh_311rest · public_data · beta
ramprest · erp · beta
renton_311rest · public_data · beta
rickandmortyapirest · entertainment · stable
sacramento_311rest · public_data · beta
salesforcerest · saas · beta
sanantonio_311rest · public_data · beta
sanfrancisco_311rest · public_data · beta
sangabriel_311rest · public_data · beta
sanjoseca_311rest · public_data · beta
sanmateocounty_311rest · public_data · beta
sanrafael_311rest · public_data · beta
santafe_311rest · public_data · beta
scottsdale_311rest · public_data · beta
seattle_311rest · public_data · beta
shopifygraphql · ecommerce · beta
somervillema_311rest · public_data · stable
springfieldmo_311rest · public_data · beta
stpetersburg_311rest · public_data · beta
tacoma_311rest · public_data · beta
toledo_311rest · public_data · beta
topeka_311rest · public_data · beta
vallejo_311rest · public_data · beta
washingtondc_311rest · public_data · beta
waynecountymi_311rest · public_data · beta
westdesmoines_311rest · public_data · beta
wichita_311rest · public_data · beta
worcester_311rest · public_data · beta
youngstown_311rest · public_data · beta
128 certified, and 881 more vendors Ingest builds on request. Browse them all.
API kinds
File
Buckets and local paths via fsspec: `csv`, `jsonl`, `json`, `parquet`, `xlsx`.
GraphQL
One document per resource, POSTed as `{query, operationName, variables}`.
gRPC
A channel per resource, TLS by default, with auth carried in metadata.
REST
Ingest's original engine. Only `{method,url,headers,params,data,json,timeout,allow_redirects,ssl,proxy}` are forwarded to the transport.
SOAP
Envelope templates rendered with XML-escaped parameters, parsed with `lxml`.
SQL
Databases read through DB-API or ADBC with chunked fetch.
WebSocket
A bounded batch window over a stream, not real-time delivery.
Destinations
Amazon AthenaIceberg tables in S3 and the AWS Glue Data Catalog, ready to query with Athena. Supports append, replace, and merge.
BigQuery
Load jobs straight from the worker. Native `MERGE`. No bucket to provision.
ClickHouse
Native insert. Merge uses `ReplacingMergeTree` keyed by `dedup_sort`.
Databricks
Loads through a Unity Catalog volume into Delta tables. Native MERGE for upserts.
DuckDB
Not a hosted destination. For DuckDB, use MotherDuck, or keep Iceberg tables in your own S3 bucket and query them with DuckDB.
ABAzure Blob Storage
Parquet files in your Azure container, authorised by a connection string.
Google Cloud Storage
Parquet files in your own GCS bucket. Ingest holds no credential: your runs authenticate with the AWS identity your workspace already has.
Apache Iceberg
Iceberg tables in your REST catalog. Catalog and storage authentication must be supported by the setup form.
Amazon S3
Parquet files in your S3 bucket, written under a role you own and can revoke.
Generic SQL
MySQL with batched inserts and transactional merge emulation. Review the current TLS setup limitation before connecting.
MotherDuckDuckDB, hosted. Same adapter, same semantics.
SQL ServerNot yet available as an Ingest destination.
Postgres
The conformance oracle. `ON CONFLICT` merge, transactional.
RedshiftCOPY from your S3 staging prefix into Redshift, with native MERGE for upserts.
Snowflake
Tables in your Snowflake database, authenticated with your Ingest workspace identity. No stored Snowflake key or password.