REST
Ingest's original engine. Only `{method,url,headers,params,data,json,timeout,allow_redirects,ssl,proxy}` are forwarded to the transport.
Errors are detected in band
A 4xx that is not 429 is an error. So is a 2xx whose body is not JSON (the first 200 characters are
recorded), and so is a 2xx whose body matches the resource's error_path / error_conditions. An API
that answers 200 OK with {"ok": false} has failed, and Ingest treats it that way.
Pagination
none | offset | page | cursor | link | next_url_in_body. Every paginator declares a stop condition.
Offset paging always re-emits the limit parameter on later pages; page-numbered APIs use page,
never offset; link follows RFC 5988 with a re-merged-params self-heal for servers that 400 or 422
on a bare next link. Stop precedence is has_more_path first, then a short page. Hard caps: 1000
pages, 500,000 records per work item. A response that switches between dict and list is fatal.
Credential proof
A trivial authenticated call. 404, 405 and 410 are not credential verdicts: they mean the path was wrong, not the key.
Questions
- Which REST sources has Ingest certified?
- Abilene TX 311, Albany NY 311, Ann Arbor MI 311, Anne Arundel County MD 311, Anthropic, Asana, Auburn WA 311, Aurora IL 311, Austin TX 311, Baltimore MD 311, Baton Rouge LA 311, Beaumont TX 311, Bellevue WA 311, Bellingham WA 311, Berkeley CA 311, Bethlehem PA 311, Bloomington IN 311, Boca Raton FL 311, Boston MA 311, Bridgeport CT 311, Brockton MA 311, Brookline MA 311, Buffalo NY 311, Burlington VT 311, Cambridge MA 311, Cambridge MA Open Data, Charleston County SC 311, Charlotte NC 311, Chula Vista CA 311, Cincinnati OH 311, Chicago IL 311, New York City NY 311, Clayton County GA 311, Cleveland OH 311, Cobb County GA 311, Columbus OH 311, Corona CA 311, Dallas TX 311, Davenport IA 311, Denver CO 311, Detroit MI 311, Elgin IL 311, Encinitas CA 311, Everett WA 311, Federal Way WA 311, Fort Lauderdale FL 311, Gilbert AZ 311, Google Ad Manager, Google Analytics 4, Greenwich CT 311, Hartford CT 311, Honolulu HI 311, Houston TX 311, Howard County MD 311, Huntington WV 311, Huntsville AL 311, Indianapolis IN 311, Jersey City NJ 311, Jira, Kansas City MO 311, Kent WA 311, Knox County TN 311, Los Angeles CA 311, Lexington KY 311, Lincoln NE 311, LinkedIn Ads, Livonia MI 311, Louisville KY 311, Lubbock TX 311, Macon-Bibb County GA 311, Manchester NH 311, Maui County HI 311, Medford MA 311, Memphis TN 311, Miami FL 311, Miami-Dade County FL 311, Minneapolis MN 311, Montgomery County MD 311, Multnomah County OR 311, Naperville IL 311, Nashua NH 311, Nashville TN 311, Newark NJ 311, NOAA, New Orleans LA 311, Norfolk VA 311, Oakland CA 311, Omaha NE 311, Open Brewery DB, Paterson NJ 311, Peoria IL 311, Philadelphia PA 311, Pinellas County FL 311, Pittsburgh PA 311, PokéAPI, Portland ME 311, Prince George's County MD 311, Provo UT 311, Raleigh NC 311, Ramp, Renton WA 311, The Rick and Morty API, Sacramento CA 311, Salesforce, San Antonio TX 311, San Francisco CA 311, San Gabriel CA 311, San Jose CA 311, San Mateo County CA 311, San Rafael CA 311, Santa Fe NM 311, Scottsdale AZ 311, Seattle WA 311, Somerville MA 311, Springfield MO 311, St. Petersburg FL 311, Tacoma WA 311, Toledo OH 311, Topeka KS 311, Vallejo CA 311, Washington DC 311, Wayne County MI 311, West Des Moines IA 311, Wichita KS 311, Worcester MA 311 and Youngstown OH 311. Each one has run against the real API, with exact row counts asserted.
- Can Ingest sync any REST API?
- Ingest authors the connector from the vendor's published API documentation, runs it against the real API, and lists it as deployable only once it passes. A connector is a declarative spec, never per-source code, which is why a new one is authored rather than programmed.
- Where can REST data land?
- Amazon S3, Google Cloud Storage, Azure Blob Storage, Apache Iceberg, Ingest Managed Lakehouse, MotherDuck, Postgres, Amazon Athena, Databricks, Redshift, BigQuery, ClickHouse, MySQL / generic SQL and Snowflake. The data lands in a destination you own, under credentials you control. Ingest keeps no copy of it.
REST sources, and where they land
Certified REST connectors: Abilene TX 311, Albany NY 311, Ann Arbor MI 311, Anne Arundel County MD 311, Anthropic, Asana, Auburn WA 311, Aurora IL 311, Austin TX 311, Baltimore MD 311, Baton Rouge LA 311, Beaumont TX 311, Bellevue WA 311, Bellingham WA 311, Berkeley CA 311, Bethlehem PA 311, Bloomington IN 311, Boca Raton FL 311, Boston MA 311, Bridgeport CT 311, Brockton MA 311, Brookline MA 311, Buffalo NY 311, Burlington VT 311, Cambridge MA 311, Cambridge MA Open Data, Charleston County SC 311, Charlotte NC 311, Chula Vista CA 311, Cincinnati OH 311, Chicago IL 311, New York City NY 311, Clayton County GA 311, Cleveland OH 311, Cobb County GA 311, Columbus OH 311, Corona CA 311, Dallas TX 311, Davenport IA 311, Denver CO 311, Detroit MI 311, Elgin IL 311, Encinitas CA 311, Everett WA 311, Federal Way WA 311, Fort Lauderdale FL 311, Gilbert AZ 311, Google Ad Manager, Google Analytics 4, Greenwich CT 311, Hartford CT 311, Honolulu HI 311, Houston TX 311, Howard County MD 311, Huntington WV 311, Huntsville AL 311, Indianapolis IN 311, Jersey City NJ 311, Jira, Kansas City MO 311, Kent WA 311, Knox County TN 311, Los Angeles CA 311, Lexington KY 311, Lincoln NE 311, LinkedIn Ads, Livonia MI 311, Louisville KY 311, Lubbock TX 311, Macon-Bibb County GA 311, Manchester NH 311, Maui County HI 311, Medford MA 311, Memphis TN 311, Miami FL 311, Miami-Dade County FL 311, Minneapolis MN 311, Montgomery County MD 311, Multnomah County OR 311, Naperville IL 311, Nashua NH 311, Nashville TN 311, Newark NJ 311, NOAA, New Orleans LA 311, Norfolk VA 311, Oakland CA 311, Omaha NE 311, Open Brewery DB, Paterson NJ 311, Peoria IL 311, Philadelphia PA 311, Pinellas County FL 311, Pittsburgh PA 311, PokéAPI, Portland ME 311, Prince George's County MD 311, Provo UT 311, Raleigh NC 311, Ramp, Renton WA 311, The Rick and Morty API, Sacramento CA 311, Salesforce, San Antonio TX 311, San Francisco CA 311, San Gabriel CA 311, San Jose CA 311, San Mateo County CA 311, San Rafael CA 311, Santa Fe NM 311, Scottsdale AZ 311, Seattle WA 311, Somerville MA 311, Springfield MO 311, St. Petersburg FL 311, Tacoma WA 311, Toledo OH 311, Topeka KS 311, Vallejo CA 311, Washington DC 311, Wayne County MI 311, West Des Moines IA 311, Wichita KS 311, Worcester MA 311, Youngstown OH 311.
The other source kinds Ingest reads: File, GraphQL, gRPC, SOAP, SQL, WebSocket.
Where the tables land: Amazon S3, Google Cloud Storage, Azure Blob Storage, Apache Iceberg, Ingest Managed Lakehouse, MotherDuck, Postgres, Amazon Athena, Databricks, Redshift, BigQuery, ClickHouse, MySQL / generic SQL, Snowflake.