|
|
|
# Architecture
|
|
|
|
|
|
|
|
## System Overview
|
|
|
|
|
|
|
|
The Automated Email Migration System consists of several key components working together to automate the email migration process.
|
|
|
|
|
|
|
|
## User Migration Journey
|
|
|
|
|
|
|
|
This diagram shows how a SabahGov user goes through the email migration process:
|
|
|
|
|
|
|
|
```
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ USER EMAIL MIGRATION JOURNEY │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
|
|
|
|
STEP 1: USER CONSENT
|
|
|
|
│
|
|
|
|
User fills consent form on Bohrium
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
Consent stored in Bohrium database
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 2: QUEUE SYNC (1:00 PM Daily) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
System fetches user consent from Bohrium API
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
User added to migration queue
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
System automatically creates batch and assigns to server
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 3: PRE-MIGRATION EMAIL (1:30 PM Daily) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
User receives email notification:
|
|
|
|
- Migration date and time
|
|
|
|
- What to expect
|
|
|
|
- Instructions
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 4: AD GROUP MANAGEMENT (5:30 PM Daily) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
User added to: sabahgov-gws-gmail (Exchange Security Group)
|
|
|
|
│
|
|
|
|
User removed from: sabahgov-non-gmail (Exchange Distribution Group)
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 5: GCDS SYNC (5:30 PM - After AD Management) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
GCDS synchronizes Active Directory changes to Google Workspace
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
User appears in sabahgov-gws OU in Google Workspace
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 6: PST EXPORT (6:00 PM Daily) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
On Hafnium Server:
|
|
|
|
│
|
|
|
|
├─► Disable-Mailbox (Exchange mailbox disabled)
|
|
|
|
│ │
|
|
|
|
│ ▼
|
|
|
|
├─► Enable-MailUser (Convert to MailUser)
|
|
|
|
│ │
|
|
|
|
│ ▼
|
|
|
|
└─► Export mailbox to PST file
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
PST file saved to network share (Z:\TEMP01-PST, Z:\TEMP02-PST, etc.)
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 7: GWME MIGRATION (6:00 PM - After PST Export) │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
On Migration Server (TEMP01/TEMP02/TEMP03):
|
|
|
|
│
|
|
|
|
├─► GWME reads PST file
|
|
|
|
│ │
|
|
|
|
│ ▼
|
|
|
|
├─► GWME imports email data to Google Workspace
|
|
|
|
│ │
|
|
|
|
│ ▼
|
|
|
|
└─► User mailbox created in Google Workspace
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
All emails, contacts, calendar migrated
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
|
|
│ STEP 8: COMPLETION │
|
|
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
│
|
|
|
|
User can now access email in Google Workspace
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
Migration status updated in database
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
Master summary log updated
|
|
|
|
```
|
|
|
|
|
|
|
|
## Detailed PST Export Process
|
|
|
|
|
|
|
|
The PST export step includes important Exchange operations:
|
|
|
|
|
|
|
|
```
|
|
|
|
PST EXPORT DETAILED STEPS:
|
|
|
|
│
|
|
|
|
├─► 1. Disable-Mailbox
|
|
|
|
│ - Exchange mailbox is disabled
|
|
|
|
│ - User can no longer access Exchange mailbox
|
|
|
|
│ - Mailbox data remains for export
|
|
|
|
│
|
|
|
|
├─► 2. Enable-MailUser
|
|
|
|
│ - Convert disabled mailbox to MailUser
|
|
|
|
│ - MailUser can receive external emails
|
|
|
|
│ - Preserves email address and routing
|
|
|
|
│
|
|
|
|
├─► 3. Export to PST
|
|
|
|
│ - Export all mailbox data (emails, folders, items)
|
|
|
|
│ - Create PST file on network share
|
|
|
|
│ - PST file ready for GWME import
|
|
|
|
│
|
|
|
|
└─► 4. Update Offline Address Book
|
|
|
|
- Update Exchange address book
|
|
|
|
- Reflect MailUser changes
|
|
|
|
```
|
|
|
|
|
|
|
|
## Components
|
|
|
|
|
|
|
|
### Core Services
|
|
|
|
|
|
|
|
- **Daily Scheduler** (`daily_scheduler.py`) - Orchestrates all daily tasks
|
|
|
|
- **Orchestrator** (`orchestrator.py`) - Coordinates migration execution
|
|
|
|
- **Remote Executor** (`remote_executor.py`) - Handles remote PowerShell execution
|
|
|
|
- **Xenon Executor** (`xenon_executor.py`) - Manages GCDS synchronization
|
|
|
|
- **Batch Manager** (`batch_manager.py`) - Manages migration batches
|
|
|
|
- **AD Group Manager** (`ad_group_manager.py`) - Handles Active Directory group operations
|
|
|
|
|
|
|
|
### Data Layer
|
|
|
|
|
|
|
|
- **Database** (`database.py`) - SQLite database for queue and batch management
|
|
|
|
- **Server Registry** (`server_registry.py`) - Migration server configuration
|
|
|
|
|
|
|
|
### Integration
|
|
|
|
|
|
|
|
- **Bohrium Client** (`bohrium_client.py`) - Fetches user consents from Bohrium API
|
|
|
|
- **GWME Handler** (`gwme_handler.py`) - Google Workspace Migration for Exchange integration
|
|
|
|
- **GAM Executor** (`gam_executor.py`) - Google Apps Manager operations
|
|
|
|
- **PST Export Manager** (`pst_export_manager.py`) - PST file export management
|
|
|
|
|
|
|
|
## Infrastructure
|
|
|
|
|
|
|
|
### Servers
|
|
|
|
|
|
|
|
- **TEMP01** - Main application server (Flask dashboard + scheduler)
|
|
|
|
- **TEMP02/TEMP03** - Migration execution servers (GWME)
|
|
|
|
- **Hafnium** - Remote PowerShell execution server (PST export)
|
|
|
|
- **Xenon** - GCDS synchronization server
|
|
|
|
|
|
|
|
## Data Flow
|
|
|
|
|
|
|
|
1. **Queue Sync** - Fetch consents from Bohrium API → Add to migration queue
|
|
|
|
2. **Batch Creation** - Create batches and assign to servers
|
|
|
|
3. **Pre-Migration** - Send notification emails to users
|
|
|
|
4. **AD Management** - Update Active Directory groups
|
|
|
|
5. **GCDS Sync** - Synchronize AD with Google Workspace
|
|
|
|
6. **PST Export** - Export mailboxes to PST files
|
|
|
|
7. **GWME Migration** - Import PST files to Google Workspace
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
**See also:** [System Components](System-Components.md), [Workflow](Workflow.md)
|
|
|
|
|
