What is Coral?
Coral is an open-source tool that lets you query any API, database,
or file as SQL — no ETL, no warehouse, no glue code.
The Problem I Solved
dbt Cloud is used by thousands of data engineers. But there was no
way to query dbt Cloud jobs, runs, and environments via SQL. I built that.
What I Built
PR #627 — dbt Cloud Community Source
Added 3 tables to Coral:
-
dbt_cloud.jobs— all jobs with schedules and state -
dbt_cloud.runs— run history with status codes -
dbt_cloud.environments— deployment environments
The Code
# manifest.yaml (simplified)
name: dbt_cloud
backend: http
tables:
- name: jobs
- name: runs
- name: environments
Live Demo
-- Query 890 open PRs from Coral repo
SELECT number, title, state
FROM github.pulls
WHERE owner='withcoral' AND repo='coral'
AND state='open'
LIMIT 1000
Result: 890 rows in ~10 seconds. No ETL. Just SQL.
Status Codes I Learned
dbt Cloud run status codes:
- 1 = Queued
- 2 = Starting
- 3 = Running
- 10 = Success
- 20 = Error
- 30 = Cancelled
Challenges
- Pagination — dbt Cloud uses offset pagination, had to implement correctly
- Timestamps — ISO8601 format needed special handling in Coral DSL
- Virtual columns — filter echo columns needed for debugging
What I Learned
- How to write a Coral source manifest (DSL v3)
- HTTP backend configuration
- Open source contribution workflow — fork, branch, PR, review
- Real API pagination patterns
The Result
PR #627 merged into withcoral/coral ✅
Now any Coral user can:
SELECT j.name, r.status, r.started_at
FROM dbt_cloud.runs r
JOIN dbt_cloud.jobs j ON j.id = r.job_id
WHERE r.status = 20 -- Error
ORDER BY r.started_at DESC
LIMIT 20;
Try It Yourself
coral source add --file sources/community/dbt_cloud/manifest.yaml
coral sql "SELECT * FROM dbt_cloud.jobs LIMIT 10"
Links
- Merged PR: https://github.com/withcoral/coral/pull/627
- Coral repo: https://github.com/withcoral/coral
- Pirates of the Coral-bean hackathon: https://wemakedevs.org/hackathons/coral
Built during Pirates of the Coral-bean hackathon by @WeMakeDevs 🏴☠️
