What You'll Need
- n8n Cloud or self-hosted n8n instance
- Hetzner VPS or Contabo VPS for self-hosted deployments
- DigitalOcean as an alternative hosting provider
- Python 3.8+ (for Airflow)
- Basic API knowledge and familiarity with REST endpoints
Table of Contents
- The Core Difference: When to Pick Each
- n8n for Rapid API Workflows
- Airflow for Complex DAGs
- Hands-On: Building the Same Workflow in Both
- Performance, Scaling, and Real Costs
- Getting Started
The Core Difference: When to Pick Each
I've spent the last three years automating workflows with both n8n and Airflow, and I can tell you straight: they're built for different problems.
n8n is a no-code/low-code automation platform. You drag nodes, connect them visually, and deploy workflows in minutes. It's HTTP-first, has 400+ integrations baked in, and you don't need to write Python. Perfect for startups, small teams, and anyone who values speed.
Airflow is a data orchestration engine built by Airbnb. It's code-centric, Python-native, and designed for complex DAGs (Directed Acyclic Graphs). You write operators, define task dependencies, and gain granular control. It scales to thousands of jobs per second but requires engineering muscle.
The hard truth: if your workflow fits in 100 nodes and calls 5 APIs, n8n wins on velocity. If you're orchestrating 50 data pipelines with retry logic, monitoring, and multi-environment deployments, Airflow is built for that war.
Let me show you both in action.
n8n for Rapid API Workflows
I'll build a real example: a workflow that pulls GitHub repository stats, enriches them with API data, and saves results to a database. With n8n, this takes 15 minutes.
Step 1: Set Up n8n Cloud or Self-Hosted
If you're using n8n Cloud, sign up and skip to Step 2. If self-hosting, grab a Hetzner VPS with 2GB RAM ($4/month) or Contabo VPS ($3/month). SSH in and run:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
mkdir n8n && cd n8n
Create a docker-compose.yml:
version: '3.8'
services:
n8n:
image: n8nio/n8n
container_name: n8n
ports:
- "5678:5678"
environment:
- N8N_HOST=your-domain.com
- N8N_PROTOCOL=https
- NODE_ENV=production
- WEBHOOK_TUNNEL_URL=https://your-domain.com/
- DB_TYPE=postgres
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=n8n
- DB_POSTGRESDB_USER=n8n
- DB_POSTGRESDB_PASSWORD=securepassword123
depends_on:
- postgres
volumes:
- n8n_data:/home/node/.n8n
networks:
- n8n-network
restart: unless-stopped
postgres:
image: postgres:15-alpine
container_name: n8n-postgres
environment:
- POSTGRES_DB=n8n
- POSTGRES_USER=n8n
- POSTGRES_PASSWORD=securepassword123
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- n8n-network
restart: unless-stopped
volumes:
n8n_data:
postgres_data:
networks:
n8n-network:
driver: bridge
Start it:
sudo docker-compose up -d
Access via http://localhost:5678 (or your domain after SSL setup with Nginx).
Step 2: Create the GitHub Stats Workflow
In n8n, create a new workflow:
- Trigger: Click the "+" button, add a Webhook node set to POST
-
GitHub Node: Search "GitHub" in the node library, add it
- Authentication: Select OAuth or add your GitHub token
- Resource: "Repository"
- Operation: "Get"
- Owner:
{{ $json.owner }} - Repository:
{{ $json.repo }}
-
HTTP Request Node: For additional API enrichment
- Method: GET
- URL:
https://api.github.com/repos/{{ $json.owner }}/{{ $json.repo }}/traffic/popular - Headers: Add
Authorization: token {{ $env.GITHUB_TOKEN }}
-
Data Mapper Node: Transform the response
- Map output to
name,stars,forks,language
- Map output to
-
PostgreSQL Node: Store results
- Operation: "Insert"
- Table:
github_stats - Columns:
repo_name,stars,forks,language,fetched_at
-
Send Slack Notification: Add a Slack node (optional)
- Message:
Fetched stats for {{ $json.name }} — {{ $json.stars }} stars
- Message:
Test with a webhook POST:
curl -X POST http://localhost:5678/webhook/github-stats \
-H "Content-Type: application/json" \
-d '{"owner":"torvalds","repo":"linux"}'
Response will show 5000+ stars and you've built a real workflow in minutes. No Python, no DAG files.
💡 Fast-Track Your Project: Don't want to configure this yourself? I build custom n8n pipelines and bots. Message me with code SYS3-DEVTO.
Airflow for Complex DAGs
Now let's build the same workflow in Airflow. It's more verbose but gives you industrial-grade control.
Step 1: Install Airflow
On a fresh DigitalOcean droplet (Ubuntu 22.04, 2GB RAM, $6/month):
python3 -m venv airflow-env
source airflow-env/bin/activate
pip install --upgrade pip
pip install apache-airflow==2.8.0
pip install apache-airflow-providers-http
pip install apache-airflow-providers-postgres
pip install apache-airflow-providers-slack
pip install requests
Initialize the database:
export AIRFLOW_HOME=~/airflow
airflow db init
airflow users create --username admin --password admin --firstname Admin --lastname User --role Admin --email admin@example.com
Start the scheduler and webserver:
airflow scheduler &
airflow webserver --port 8080 &
Access at http://localhost:8080.
Step 2: Write the DAG
Create ~/airflow/dags/github_stats_dag.py:
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.operators.http_operator import SimpleHttpOperator
from airflow.providers.postgres.operators.postgres import PostgresOperator
from airflow.providers.slack.operators.slack_webhook import SlackWebhookOperator
from airflow.models import Variable
import requests
import json
default_args = {
'owner': 'data-team',
'retries': 2,
'retry_delay': timedelta(minutes=5),
'email_on_failure': True,
'email': ['ops@example.com'],
}
dag = DAG(
'github_stats_pipeline',
default_args=default_args,
description='Fetch GitHub repo stats and store in Postgres',
schedule_interval='0 */6 * * *',
start_date=datetime(2024, 1, 1),
catchup=False,
tags=['github', 'api', 'analytics'],
)
def fetch_github_data(**context):
owner = context['dag_run'].conf.get('owner', 'torvalds')
repo = context['dag_run'].conf.get('repo', 'linux')
token = Variable.get('GITHUB_TOKEN')
headers = {'Authorization': f'token {token}'}
url = f'https://api.github.com/repos/{owner}/{repo}'
response = requests.get(url, headers=headers, timeout=10)
response.raise_for_status()
data = response.json()
context['task_instance'].xcom_push(
key='repo_data',
value={
'owner': owner,
'name': data['name'],
'stars': data['stargazers_count'],
'forks': data['forks_count'],
'language': data['language'],
'description': data['description'],
}
)
def enrich_with_traffic(**context):
task_instance = context['task_instance']
repo_data = task_instance.xcom_pull(task_ids='fetch_github', key='repo_data')
owner = repo_data['owner']
repo = repo_data['name']
token = Variable.get('GITHUB_TOKEN')
headers = {'Authorization': f'token {token}'}
url = f'https://api.github.com/repos/{owner}/{repo}/traffic/popular'
try:
response = requests.get(url, headers=headers, timeout=10)
response.raise_for_status()
traffic = response.json()
repo_data['top_paths'] = len(traffic) if isinstance(traffic, list) else 0
except:
repo_data['top_paths'] = 0
task_instance.xcom_push(key='enriched_data', value=repo_data)
def prepare_insert_statement(**context):
task_instance = context['task_instance']
enriched = task_instance.xcom_pull(task_ids='enrich_traffic', key='enriched_data')
insert_sql = f"""
INSERT INTO github_stats (repo_owner, repo_name, stars, forks, language, top_paths, fetched_at)
VALUES ('{enriched['owner']}', '{enriched['name']}', {enriched['stars']}, {enriched['forks']}, '{enriched['language']}', {enriched['top_paths']}, NOW());
"""
task_instance.xcom_push(key='insert_sql', value=insert_sql)
fetch_github = PythonOperator(
task_id='fetch_github',
python_callable=fetch_github_data,
dag=dag,
provide_context=True,
)
enrich_traffic = PythonOperator(
task_id='enrich_traffic',
python_callable=enrich_with_traffic,
dag
Originally published on Automation Insider.
