GBase 8c Distributed Cluster Data Migration and Synchronization: A Cross‑Version, Cross‑Site, Cross‑Cloud Guide

Migrating a GBase 8c distributed cluster involves shard distribution, version compatibility, and strict data consistency. This guide, drawn from production experience, covers pre‑migration preparation, three core migration strategies, post‑migration validation, and common failure recovery — with ready‑to‑use commands for your gbase database.

1. Pre‑Migration Essentials

1.1 Scope Definition and Risk Assessment

Identify objects: Tables, indexes, views, stored procedures, sequences, synonyms — check target version support.
Risk plan: Define rollback strategy, schedule migration during off‑peak hours (0:00–3:00).

1.2 Environment Compatibility Checklist

Check Item	Key Point	Command / Operation
Version compatibility	Confirm direct upgrade possible, or require intermediate version	`gs_om -t version`
Architecture match	CN/DN count, shard strategy match	`SELECT * FROM pg_dist_shard;`, `gs_om -t status --detail`
Hardware / network	30% resource headroom, cross‑site latency ≤50ms	`top`, `df -h`, `ping`
Privileges and storage	Create dedicated migration user with SUPERUSER, test storage read/write	`CREATE ROLE mig_user WITH LOGIN SUPERUSER PASSWORD 'xxx';`

1.3 Backup (Mandatory)

# Full backup (data + config + metadata)
gs_basebackup -D /backup/gbase8c/mig_$(date +%Y%m%d) \
  -h <source_CN_IP> -p 5432 -U mig_user \
  -F tar -X stream -P -v

# Verify backup integrity
gs_basebackup -C -D /backup/gbase8c/mig_$(date +%Y%m%d) \
  -h <source_CN_IP> -p 5432 -U mig_user -v

Backup size should be at least 1.2× the source data. Store on a separate device.

1.4 Service Quiesce

Suspend write operations; keep read access.
Disable source cluster cron jobs and backup tasks.
Lock large tables if necessary: LOCK TABLE table_name IN ACCESS SHARE MODE;

2. Three Core Migration Strategies

2.1 Strategy 1: Cross‑Version Upgrade (e.g., V8.2→V8.4)

Flow: Backup → upgrade target cluster → full migration → incremental sync → validate → cutover.

Upgrade target cluster

   cd /GBase_HOME/script/upgrade
   ./gs_upgrade -s <source_install_dir> -d <target_install_dir> \
     -U mig_user -D <source_data_dir> -D <target_data_dir>

Verify: gs_om -t version.

Full data migration

   gs_om -t stop
   gs_basebackup -R -D <target_data_dir> -h <target_CN_IP> -p 5432 -U mig_user \
     -f /backup/gbase8c/mig_20260331/full_backup.tar -F tar
   gs_om -t start

Incremental sync Enable WAL archiving on source; apply WALs on target with pg_waldump.

2.2 Strategy 2: Cross‑Site / Cross‑Cloud Cluster Migration

Core: Physical backup + incremental sync + DNS / pool switch.

Full backup and transfer

   rsync -avP /backup/gbase8c/mig_20260331/ <target_IP>:/backup/gbase8c/mig_20260331/

Restore and initialise target cluster

Recover full backup, configure pg_hba.conf to allow application access.
Incremental sync and cutover

Switch traffic by updating DNS or connection pool to point to target CN. Always validate data consistency before cutover.

2.3 Strategy 3: Single Table / Shard Migration

Tools: gs_dump / gs_restore / COPY.

# Export a single table
gs_dump -h <source_CN_IP> -p 5432 -U mig_user -d gbase -t order -F tar -f /backup/order_dump.tar

# Import to target
gs_restore -h <target_CN_IP> -p 5432 -U mig_user -d gbase -F tar /backup/order_dump.tar

For large tables (>100GB), prefer COPY:

COPY "order" TO '/data/order_data.csv' WITH CSV HEADER;
-- After transferring the file to target
COPY "order" FROM '/data/order_data.csv' WITH CSV HEADER;

3. Post‑Migration Validation

3.1 Data Consistency Checks

Row counts: SELECT COUNT(*) FROM order;
Shard consistency: gs_om -t status --detail | grep "shard" and run gs_sync_check.
Spot‑check latest 100 records between source and target.

3.2 Performance and Functional Tests

Core business SQL response time ≥ 90% of source.
Business flows (order, payment) run without errors.
CPU/Memory/IO ≤ 80%, no abnormal alerts.

3.3 Service Cutover and Monitoring

Re‑enable writes and cron jobs.
Set up Prometheus+Grafana monitoring for node status, shard sync, and resource usage.
Configure full+incremental backups on the new cluster.

4. Common Migration Failures and Recovery

Symptom	Cause	Solution
“shard_id does not exist”	Mismatched shard strategy	Align shard definitions, re‑migrate
WAL sync lag, data inconsistency	High network latency, WAL archiving off	Optimise network, re‑enable WAL
“Permission denied” during import	Insufficient user or directory permissions	Grant SUPERUSER, `chmod 755` storage
SQL errors after upgrade	Deprecated functions/parameters	Update SQL per target version docs
Startup failure, invalid config	Wrong parameter values	Fix invalid parameters or restore default postgresql.conf

5. Migration Best Practices

Always backup – even in test runs.
Check version compatibility – consult upgrade notes before starting.
Validate, then cutover – never switch traffic before data checks.
Reserve 30% resource headroom on target.
Use shard‑by‑shard migration for large clusters – reduces per‑batch risk.
Automate with scripts – chain backup, export, import, and validation into a single Shell script to reduce human error.