AWS DMS Migration Guide

1.1: Navigate to DMS Console

1. In AWS Console search bar, type "DMS"
2. Click "Database Migration Service"
3. You'll see the DMS dashboard

1.2: Create Replication Instance

1. In left sidebar, click "Replication instances"
2. Click orange "Create replication instance" button

You'll see a form. Fill it in as follows:

1.3: Create the Instance

1. Scroll to bottom
2. Click orange "Create replication instance" button
3. Wait - You'll see a banner: "Creating replication instance..."

1.4: Note the IP Address

1. Click on your replication instance name
2. Look for "Public IP address" or "Private IP address"
3. Copy the IP address
4. Save it - you'll need it for Supabase firewall (if applicable)

2.1: Navigate to EC2 Security Groups

1. In AWS Console search bar, type "EC2"
2. Click "EC2" to open EC2 console
3. In left sidebar, scroll down to "Network & Security"
4. Click "Security Groups"

2.2: Find Aurora's Security Group

1. In the search box, enter your Aurora security group ID
2. Click on the security group name to select it

2.3: Add Inbound Rule for DMS

1. Look at bottom tabs
2. Click "Inbound rules" tab
3. Click "Edit inbound rules" button (top right)
4. Click "Add rule" button

5. Click "Save rules" button (orange, bottom right)
6. You'll see: "Successfully modified security group rules"

3.1: Navigate to Endpoints

1. In DMS Console, look at left sidebar
2. Click "Endpoints"
3. Click orange "Create endpoint" button

3.2: Configure Endpoint

3.3: Add Endpoint Settings (For CDC)

Scroll down to find "Endpoint settings":

1. Click "Add new setting" button
2. Enter this JSON:

{
  "PluginName": "pglogical",
  "HeartbeatEnable": true,
  "HeartbeatFrequency": 1
}

3.4: Create Endpoint

1. Scroll to bottom
2. Click "Run test" button (if you filled test connection details)
3. Wait 30-60 seconds
4. Look for: "Connection tested successfully" ✅
5. Click orange "Create endpoint" button

4.1: Create Endpoint

1. Still in Endpoints section, click "Create endpoint" button again

4.2: Configure Endpoint

4.3: Add Performance Settings (Optional but Recommended)

Scroll down to "Endpoint settings":

{
  "BatchApplyEnabled": true,
  "ParallelApplyThreads": 4,
  "ParallelApplyBufferSize": 100
}

4.4: Create Endpoint

1. Click "Run test" to verify connection
2. Should show: "Connection tested successfully" ✅
3. Click "Create endpoint" button
4. Verify: Status shows "Active"

5.1: Navigate to Tasks

1. In DMS Console left sidebar, click "Database migration tasks"
2. Click orange "Create task" button

5.2: Configure Task Settings

5.3: Configure Table Mappings

This tells DMS which tables to migrate.

{
  "rules": [
    {
      "rule-type": "selection",
      "rule-id": "1",
      "rule-name": "include-aim-tables",
      "object-locator": {
        "schema-name": "public",
        "table-name": "user_dagad_%"
      },
      "rule-action": "include"
    }
  ]
}

5.4: Review and Create Task

1. Scroll to bottom
2. Review all settings
3. Click orange "Create task" button
4. You'll see: "Creating database migration task..."

Main Task Dashboard

1. In DMS Console, click "Database migration tasks"
2. Find your task: supabase-to-aurora-aim-migration
3. Look at Status column:
   • "Starting": Task is initializing
   • "Running": Full load in progress
   • "Load complete": Full load done, CDC ongoing ✅
   • "Failed": Something went wrong (see logs)

Detailed Task View

1. Click on your task name
2. You'll see detailed information:
   • Status: Current state
   • % complete: Progress percentage
   • Tables loaded: Number of tables completed
   • Rows loaded: Total rows copied

View Table Statistics

1. In task details, look for tabs at top
2. Click "Table statistics" tab
3. You'll see a table with these columns:
   • Table name: Which table
   • Full load: Rows copied during initial load
   • Inserts: New rows added (CDC)
   • Updates: Rows modified (CDC)
   • Deletes: Rows removed (CDC)
   • Validation: Data validation status

Access Logs from DMS Console

1. In task details, click "Monitoring" tab
2. Scroll down to "Logs"
3. Click "View CloudWatch Logs"

Access Logs from CloudWatch

1. In AWS Console search bar, type "CloudWatch"
2. Click "CloudWatch"
3. In left sidebar, expand "Logs"
4. Click "Log groups"
5. Find and click: /aws/dms/tasks/[your-task-id]
6. Click on a log stream (usually the most recent)

View Metrics Dashboard

1. In DMS task details, click "Monitoring" tab
2. You'll see graphs for:
   • CPU utilization: Should stay under 80%
   • Free memory: Should not drop too low
   • Network receive throughput: Data coming from Supabase
   • Network transmit throughput: Data going to Aurora

Access Query Editor

1. In AWS Console, go to RDS
2. In left sidebar, look for "Query Editor"
3. Click "Query Editor"

Connect to Aurora

1. Select "Aurora" tab
2. Choose your Aurora cluster from dropdown
3. Database name: helium_production
4. Database username: admin
5. Password: [your-aurora-password]
6. Click "Connect to database"

Run Verification Queries

-- In Aurora
SELECT 'user_dagad_entries' as table_name, COUNT(*) as row_count
FROM user_dagad_entries
UNION ALL
SELECT 'user_dagad_folders', COUNT(*) FROM user_dagad_folders
UNION ALL
SELECT 'user_dagad_files', COUNT(*) FROM user_dagad_files
UNION ALL
SELECT 'user_dagad_embeddings', COUNT(*) FROM user_dagad_embeddings
UNION ALL
SELECT 'user_dagad_usage_log', COUNT(*) FROM user_dagad_usage_log
UNION ALL
SELECT 'user_dagad_addon_imports', COUNT(*) FROM user_dagad_addon_imports;

-- Get recent entries from Aurora
SELECT entry_id, user_id, title, created_at
FROM user_dagad_entries
ORDER BY created_at DESC
LIMIT 10;

-- Check for orphaned entries
SELECT COUNT(*) as orphaned_entries
FROM user_dagad_entries e
LEFT JOIN user_dagad_folders f ON e.folder_id = f.folder_id
WHERE e.folder_id IS NOT NULL AND f.folder_id IS NULL;

-- Check if embeddings exist
SELECT
    COUNT(*) as total_embeddings,
    COUNT(CASE WHEN embedding IS NOT NULL THEN 1 END) as embeddings_with_data
FROM user_dagad_embeddings;

Check CDCLatency Metric

1. Go to CloudWatch → Metrics → DMS
2. Select your task → CDCLatency
3. View graph
4. Verify: < 5 seconds ✅

Or:

1. In DMS task details, click "Monitoring" tab
2. Look at "CDC latency" graph
3. Verify: Line is near zero