# WIA-EDU-008 Digital Textbook Standard v1.2

## Phase 3: Protocol & Synchronization

**Status:** ✅ Complete
**Version:** 1.2.0
**Date:** 2025-03-15
**Philosophy:** 弘益人間 (Benefit All Humanity)
**Builds on:** v1.0 (Phase 1), v1.1 (Phase 2)

---

## 1. Overview

Phase 3 defines synchronization protocols for multi-device access, offline functionality, and conflict resolution. This ensures students can seamlessly access their textbooks across all devices, even without internet connectivity.

## 2. Synchronization Architecture

### 2.1 Sync Protocol Overview

The standard uses a hybrid approach:
- **Real-time:** WebSocket for immediate updates when online
- **Batch:** REST API for periodic sync and offline queue processing
- **Conflict Resolution:** Operational Transformation (OT) for annotations

### 2.2 Synchronization Scope

What gets synchronized:
1. Reading position (last page/location)
2. Annotations (highlights, notes, bookmarks)
3. User preferences (font size, theme, etc.)
4. Assessment responses
5. Analytics data

## 3. WebSocket Protocol

### 3.1 Connection Establishment

```javascript
const ws = new WebSocket('wss://sync.provider.com/v1/sync');

ws.onopen = function() {
  // Authenticate
  ws.send(JSON.stringify({
    type: 'auth',
    token: 'Bearer eyJhbGc...'
  }));
};
```

### 3.2 Sync Message Format

```json
{
  "type": "sync.annotation.create",
  "messageId": "msg-123456",
  "timestamp": "2025-12-25T10:30:15.234Z",
  "deviceId": "device-abc123",
  "vectorClock": {
    "device-abc123": 15,
    "device-def456": 12
  },
  "data": {
    "annotationId": "anno-789",
    "textbookId": "978-3-16-148410-0",
    "type": "highlight",
    "selector": {
      "start": 1234,
      "end": 1456
    },
    "color": "#FFFF00"
  }
}
```

### 3.3 Server Acknowledgment

```json
{
  "type": "sync.ack",
  "messageId": "msg-123456",
  "status": "success",
  "serverTime": "2025-12-25T10:30:15.567Z"
}
```

## 4. Conflict Resolution

### 4.1 Vector Clocks

Each device maintains a vector clock to track causal ordering:

```json
{
  "vectorClock": {
    "device-abc123": 15,  // This device's counter
    "device-def456": 12,  // Last seen counter from other device
    "device-ghi789": 8    // Last seen counter from third device
  }
}
```

### 4.2 Operational Transformation for Annotations

When two devices create overlapping highlights:

```javascript
// Device A creates highlight
{
  "type": "highlight",
  "range": { "start": 100, "end": 150 },
  "color": "yellow",
  "vectorClock": { "A": 5, "B": 3 }
}

// Device B creates highlight (concurrently)
{
  "type": "highlight",
  "range": { "start": 120, "end": 180 },
  "color": "green",
  "vectorClock": { "A": 4, "B": 4 }
}

// Server resolves: Keep both highlights
// Result: Two separate, overlapping highlights
```

### 4.3 Conflict Resolution Strategies

| Data Type | Strategy | Rationale |
|-----------|----------|-----------|
| Reading Position | Last-write-wins | Most recent position is most relevant |
| Highlights | Merge (keep all) | User intent is to mark multiple passages |
| Notes | Merge (keep all) | All notes have value |
| Bookmarks | Union | All bookmarks should be preserved |
| Quiz Responses | First-write-wins | Prevent duplicate submissions |
| Preferences | Last-write-wins | Most recent preference is desired |

## 5. Offline Support

### 5.1 Local Storage Requirements

Implement local-first architecture using:
- **IndexedDB** (Web): For content and user data
- **SQLite** (Native): For mobile and desktop apps
- **Sync Queue**: Persist operations when offline

### 5.2 Offline Queue Structure

```json
{
  "queueId": "queue-user123",
  "operations": [
    {
      "id": "op-001",
      "type": "annotation.create",
      "timestamp": "2025-12-25T08:15:00Z",
      "attempts": 0,
      "maxAttempts": 5,
      "data": {...},
      "dependencies": []
    },
    {
      "id": "op-002",
      "type": "progress.update",
      "timestamp": "2025-12-25T08:30:00Z",
      "attempts": 0,
      "data": {...},
      "dependencies": ["op-001"]
    }
  ]
}
```

### 5.3 Sync Process When Connectivity Returns

1. Check server connectivity
2. Authenticate/refresh tokens
3. Fetch server-side changes since last sync
4. Apply operational transformation if conflicts exist
5. Push queued local operations
6. Verify all operations acknowledged
7. Update local state with server confirmations

## 6. Content Updates and Versioning

### 6.1 Version Detection

```http
GET /textbooks/978-3-16-148410-0/version HTTP/1.1

Response:
{
  "currentVersion": "1.2.0",
  "localVersion": "1.1.0",
  "updateAvailable": true,
  "updateType": "minor",
  "releaseDate": "2025-12-20",
  "changelog": "Updated Chapter 5 with new research findings",
  "deltaSize": 2400000,
  "fullSize": 127834289
}
```

### 6.2 Delta Sync for Updates

```http
GET /textbooks/978-3-16-148410-0/delta?from=1.1.0&to=1.2.0 HTTP/1.1

Response: Binary diff (bsdiff format)
```

### 6.3 Annotation Mapping Across Versions

```json
{
  "annotationId": "anno-123",
  "originalVersion": "1.1.0",
  "newVersion": "1.2.0",
  "mapping": {
    "originalSelector": {
      "type": "TextPositionSelector",
      "start": 1234,
      "end": 1456
    },
    "mappedSelector": {
      "type": "TextPositionSelector",
      "start": 1289,
      "end": 1511
    },
    "confidence": 0.98,
    "method": "fuzzy-text-match"
  }
}
```

## 7. Bandwidth Optimization

### 7.1 Delta Sync Algorithms

Use binary diff algorithms:
- **bsdiff/bspatch**: For content files
- **JSON Patch (RFC 6902)**: For metadata and user data

### 7.2 Compression

All sync traffic MUST support:
- Gzip compression (minimum)
- Brotli compression (preferred)

Request header:
```
Accept-Encoding: br, gzip
```

### 7.3 Deduplication

Server MUST detect and eliminate redundant sync operations:
```json
{
  "operationId": "op-123",
  "hash": "sha256:abc123...",
  "deduplicated": true,
  "originalOperationId": "op-120"
}
```

## 8. Security

### 8.1 Transport Security

- **TLS 1.3** minimum for all connections
- Certificate pinning for mobile apps
- Perfect Forward Secrecy (PFS)

### 8.2 Local Data Encryption

All local data MUST be encrypted:
- **AES-256-GCM** for storage encryption
- Keys derived from user credentials via PBKDF2
- Per-device keys for multi-device scenarios

### 8.3 End-to-End Encryption (Optional)

For sensitive annotations:
```json
{
  "annotationId": "anno-456",
  "encrypted": true,
  "algorithm": "AES-256-GCM",
  "ciphertext": "base64-encoded-data",
  "iv": "initialization-vector",
  "keyId": "user-key-1"
}
```

## 9. Performance Requirements

### 9.1 Sync Latency

- Real-time sync: < 500ms (WebSocket)
- Batch sync: < 2s for typical operations
- Offline queue processing: < 5s for 100 operations

### 9.2 Bandwidth Usage

Typical sync scenarios:

| Operation | Full Sync | Delta Sync | Savings |
|-----------|-----------|------------|---------|
| 20 annotations | 125 KB | 8 KB | 94% |
| Reading progress | 125 KB | 0.5 KB | 99.6% |
| Content update (minor) | 127 MB | 2.3 MB | 98.2% |
| Preferences | 125 KB | 0.3 KB | 99.8% |

## 10. Testing and Validation

### 10.1 Sync Testing Scenarios

Test cases MUST include:
1. Concurrent edits from 2+ devices
2. Offline operation for 24+ hours
3. Network interruption during sync
4. Large annotation sets (1000+ items)
5. Content version updates
6. Conflict resolution accuracy

### 10.2 Performance Testing

- Sync 1000 annotations in < 10 seconds
- Handle 100 concurrent users per server
- Maintain < 100ms WebSocket ping latency

---

**Philosophy:** 弘益人間 · Benefit All Humanity

© 2025 WIA - World Certification Industry Association