nordabiz/docs/architecture/03-deployment-architecture.md
Maciej Pienczyn cebe52f303 refactor: Rebranding i aktualizacja modelu AI
- Zmiana nazwy: "Norda Biznes Hub" → "Norda Biznes Partner"
- Aktualizacja modelu AI: Gemini 2.0 Flash → Gemini 3 Flash
- Zachowano historyczne odniesienia w timeline i dokumentacji

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 14:08:39 +01:00

42 KiB

Deployment Architecture Diagram

Document Version: 1.0 Last Updated: 2026-01-10 Status: Production LIVE Diagram Type: Infrastructure / Deployment View


Overview

This diagram shows the physical deployment architecture of the Norda Biznes Partner system. It illustrates:

  • Physical servers and their infrastructure details (VMs, IPs, ports)
  • Network topology and connectivity
  • Service distribution across servers
  • Critical port mappings and firewall rules
  • External access points and NAT configuration

Abstraction Level: Infrastructure / Deployment Audience: DevOps, System Administrators, Infrastructure Team Purpose: Understanding physical deployment, troubleshooting connectivity issues, incident response


Deployment Architecture Diagram

graph TB
    %% External actors and entry points
    Internet["🌐 INTERNET<br/>External Users"]

    %% Public gateway
    Fortigate["🛡️ FORTIGATE FIREWALL<br/>Public IP: 85.237.177.83<br/><br/>NAT Rules:<br/>• :443 → 10.22.68.250:443<br/>• :80 → 10.22.68.250:80"]

    %% Internal network boundary
    subgraph "INPI Internal Network (10.22.68.0/24)"

        %% Reverse proxy server
        subgraph "R11-REVPROXY-01<br/>VM 119 | 10.22.68.250"
            NPM["🔒 NGINX PROXY MANAGER<br/>(Docker Container)<br/><br/>Ports:<br/>• :443 - HTTPS (SSL termination)<br/>• :80 - HTTP redirect<br/>• :81 - Admin UI (internal)<br/><br/>SSL: Let's Encrypt<br/>Certificate ID: 27<br/>Domains: nordabiznes.pl"]
        end

        %% Backend application server
        subgraph "NORDABIZ-01<br/>VM 249 | 10.22.68.249"
            subgraph "Application Layer"
                Gunicorn["🌐 GUNICORN + FLASK<br/>Port: 5000<br/>Workers: 4<br/>User: www-data<br/><br/>App: /var/www/nordabiznes<br/>Service: nordabiznes.service<br/>Timeout: 120s"]

                NginxSys["⚠️ NGINX (System)<br/>Ports: 80, 443<br/><br/>Purpose: HTTP→HTTPS redirect<br/>Status: UNUSED in production<br/><br/>⚠️ NPM should NEVER use this!"]
            end

            subgraph "Data Layer"
                PostgreSQL["💾 POSTGRESQL 14<br/>Port: 5432<br/>Listen: 127.0.0.1 ONLY<br/><br/>Database: nordabiz<br/>User: nordabiz_app<br/>Tables: 36<br/><br/>⚠️ No external connections!"]
            end

            subgraph "Background Jobs"
                Scripts["⚙️ PYTHON SCRIPTS<br/>Execution: Cron / Manual<br/><br/>• seo_audit.py<br/>• social_media_audit.py<br/>• gbp_audit.py<br/>• fetch_company_news.py"]
            end
        end

        %% Git server
        subgraph "r11-git-inpi<br/>Server | 10.22.68.180"
            Gitea["📚 GITEA<br/>Port: 3000 (HTTPS)<br/><br/>Repository: maciejpi/nordabiz<br/>SSL: Self-signed<br/>Users: maciejpi, gitadmin"]
        end
    end

    %% External services
    subgraph "External APIs (HTTPS)"
        Gemini["🤖 Google Gemini AI<br/>generativelanguage.googleapis.com<br/>Auth: API Key"]
        BraveAPI["🔍 Brave Search API<br/>api.search.brave.com<br/>Auth: API Key"]
        PageSpeed["📊 Google PageSpeed API<br/>googleapis.com/pagespeedonline<br/>Auth: API Key"]
        Places["📍 Google Places API<br/>maps.googleapis.com/maps/api<br/>Auth: API Key"]
        KRS["🏛️ KRS Open API<br/>api-krs.ms.gov.pl<br/>Auth: Public"]
        MSGraph["📧 Microsoft Graph API<br/>graph.microsoft.com<br/>Auth: OAuth 2.0"]
    end

    %% User traffic flow
    Internet -->|"HTTPS :443<br/>HTTP :80"| Fortigate
    Fortigate -->|"NAT<br/>443→443<br/>80→80"| NPM

    %% NPM to backend (CRITICAL PATH)
    NPM ==>|"⚠️ CRITICAL!<br/>HTTP :5000<br/>(NOT port 80!)"| Gunicorn
    NPM -.->|"❌ WRONG!<br/>Creates redirect loop"| NginxSys

    %% Application to database (localhost only)
    Gunicorn <-->|"SQL<br/>localhost:5432<br/>SQLAlchemy ORM"| PostgreSQL
    Scripts <-->|"SQL<br/>127.0.0.1:5432<br/>Direct connection"| PostgreSQL

    %% External API calls
    Gunicorn -->|"HTTPS<br/>API calls"| Gemini
    Gunicorn -->|"HTTPS<br/>API calls"| BraveAPI
    Gunicorn -->|"HTTPS<br/>API calls"| Places
    Gunicorn -->|"HTTPS<br/>API calls"| KRS
    Gunicorn -->|"HTTPS<br/>OAuth 2.0"| MSGraph
    Scripts -->|"HTTPS<br/>Audit requests"| PageSpeed
    Scripts -->|"HTTPS<br/>News search"| BraveAPI

    %% Git operations
    Gunicorn -.->|"git pull<br/>(deployment)"| Gitea

    %% Styling
    classDef serverStyle fill:#2c3e50,stroke:#34495e,color:#ecf0f1,stroke-width:3px
    classDef appStyle fill:#1168bd,stroke:#0b4884,color:#ffffff,stroke-width:3px
    classDef dbStyle fill:#438dd5,stroke:#2e6295,color:#ffffff,stroke-width:3px
    classDef proxyStyle fill:#e74c3c,stroke:#c0392b,color:#ffffff,stroke-width:4px
    classDef firewallStyle fill:#f39c12,stroke:#d68910,color:#ffffff,stroke-width:4px
    classDef externalStyle fill:#95a5a6,stroke:#7f8c8d,color:#ffffff,stroke-width:2px
    classDef warningStyle fill:#e67e22,stroke:#d35400,color:#ffffff,stroke-width:3px

    class NPM proxyStyle
    class Fortigate firewallStyle
    class Gunicorn,Scripts appStyle
    class PostgreSQL dbStyle
    class NginxSys warningStyle
    class Gitea serverStyle
    class Gemini,BraveAPI,PageSpeed,Places,KRS,MSGraph externalStyle

Infrastructure Inventory

Production Servers

Server VM ID IP Address Hostname OS vCPU RAM Disk Hypervisor
NORDABIZ-01 249 10.22.68.249 nordabiz-01 Ubuntu 22.04 4 8 GB 100 GB SSD Proxmox VE
R11-REVPROXY-01 119 10.22.68.250 r11-revproxy-01 Ubuntu 22.04 2 4 GB 50 GB SSD Proxmox VE
r11-git-inpi - 10.22.68.180 r11-git-inpi Ubuntu 22.04 2 4 GB 100 GB SSD Proxmox VE

Server Details

NORDABIZ-01 (Backend Application Server)

Infrastructure:

  • VM ID: 249
  • IP Address: 10.22.68.249
  • Internal DNS: nordabiznes.inpi.local
  • OS: Ubuntu 22.04 LTS
  • Resources: 4 vCPU, 8 GB RAM, 100 GB SSD
  • Hypervisor: Proxmox VE

Services Running:

Service Port Binding User Status Purpose
Gunicorn/Flask 5000 0.0.0.0 www-data Active Main Application
PostgreSQL 14 5432 127.0.0.1 postgres Active Database
Nginx (system) 80, 443 0.0.0.0 root Active HTTP→HTTPS redirect ⚠️
SSH 22 0.0.0.0 - Active Remote administration

Application Paths:

/var/www/nordabiznes/           # Application root
├── app.py                      # Main Flask application (13,144 lines)
├── database.py                 # SQLAlchemy models (36 tables)
├── venv/                       # Python virtual environment
├── templates/                  # Jinja2 templates
├── static/                     # CSS, JS, images
├── scripts/                    # Background jobs
└── .env                        # Environment variables (secrets)

/var/log/nordabiznes/           # Application logs
├── access.log                  # Gunicorn access log
└── error.log                   # Gunicorn error log

/etc/systemd/system/
└── nordabiznes.service         # Systemd unit file

Service Configuration:

# Service management
sudo systemctl status nordabiznes
sudo systemctl restart nordabiznes
sudo systemctl start nordabiznes
sudo systemctl stop nordabiznes

# View logs
sudo journalctl -u nordabiznes -f          # Follow logs
sudo journalctl -u nordabiznes -n 100      # Last 100 lines
tail -f /var/log/nordabiznes/access.log    # Access log
tail -f /var/log/nordabiznes/error.log     # Error log

SSH Access:

ssh maciejpi@10.22.68.249
# CRITICAL: Always use 'maciejpi' user, NEVER 'root'!

Gunicorn Configuration:

# /etc/systemd/system/nordabiznes.service
[Service]
ExecStart=/var/www/nordabiznes/venv/bin/gunicorn \
    --bind 0.0.0.0:5000 \
    --workers 4 \
    --timeout 120 \
    --access-logfile /var/log/nordabiznes/access.log \
    --error-logfile /var/log/nordabiznes/error.log \
    app:app

⚠️ WARNING - System Nginx:

  • System nginx on ports 80/443 is for HTTP→HTTPS redirects ONLY
  • NEVER configure NPM to use port 80 or 443 on this server
  • Doing so causes infinite redirect loop (ERR_TOO_MANY_REDIRECTS)
  • See: docs/INCIDENT_REPORT_20260102.md

R11-REVPROXY-01 (Reverse Proxy Server)

Infrastructure:

  • VM ID: 119
  • IP Address: 10.22.68.250
  • OS: Ubuntu 22.04 LTS
  • Resources: 2 vCPU, 4 GB RAM, 50 GB SSD
  • Hypervisor: Proxmox VE

Services Running:

Service Port Binding Type Purpose
NPM (HTTPS) 443 0.0.0.0 Docker Public HTTPS traffic (SSL termination)
NPM (HTTP) 80 0.0.0.0 Docker HTTP→HTTPS redirect
NPM Admin UI 81 0.0.0.0 Docker NPM management interface (internal only)
SSH 22 0.0.0.0 System Remote administration

Docker Setup:

# NPM container details
Container Name: nginx-proxy-manager_app_1
Image: jc21/nginx-proxy-manager:latest
Volumes:
  - /docker/npm/data:/data
  - /docker/npm/letsencrypt:/etc/letsencrypt

# Container management
docker ps | grep nginx-proxy-manager                    # Check status
docker logs nginx-proxy-manager_app_1 --tail 50        # View logs
docker exec -it nginx-proxy-manager_app_1 /bin/sh      # Shell access
docker restart nginx-proxy-manager_app_1               # Restart

NPM Configuration Database:

  • Location: /data/database.sqlite (inside container)
  • Access: SQLite database
  • Backup: Manual via docker cp

NPM Web UI:

SSH Access:

ssh maciejpi@10.22.68.250

Critical Proxy Host Configuration (ID: 27):

-- Query NPM database
docker exec nginx-proxy-manager_app_1 \
  sqlite3 /data/database.sqlite \
  "SELECT id, domain_names, forward_host, forward_port FROM proxy_host WHERE id = 27;"

-- Expected output:
-- 27|["nordabiznes.pl","www.nordabiznes.pl"]|10.22.68.249|5000

⚠️ CRITICAL NPM CONFIGURATION:

Parameter Value Critical Notes
Proxy Host ID 27 Fixed identifier
Domain Names nordabiznes.pl, www.nordabiznes.pl Both variants
Forward Scheme http SSL terminated at NPM
Forward Host 10.22.68.249 NORDABIZ-01 IP
Forward Port 5000 MUST BE 5000, NOT 80! ⚠️
SSL Certificate Let's Encrypt (ID 27) Auto-renewal enabled
Force SSL Yes HTTP→HTTPS redirect
HTTP/2 Yes Performance optimization
HSTS Yes max-age=31536000
Block Exploits Yes Security hardening

Verification After NPM Changes:

curl -I https://nordabiznes.pl/health
# Expected: HTTP/2 200 OK
# If redirect loop: Check forward_port is 5000!

r11-git-inpi (Git Repository Server)

Infrastructure:

  • IP Address: 10.22.68.180
  • Hostname: r11-git-inpi
  • OS: Ubuntu 22.04 LTS
  • Resources: 2 vCPU, 4 GB RAM, 100 GB SSD

Services Running:

Service Port Protocol Access Purpose
Gitea 3000 HTTPS Internal Git repository hosting
SSH 22 SSH Internal Server administration

Gitea Configuration:

User Accounts:

  • maciejpi - Personal account (repository owner)
  • gitadmin - Gitea administrator

Production Deployment via Git:

# On NORDABIZ-01
cd /var/www/nordabiznes
sudo -u www-data git -c http.sslVerify=false pull origin master
sudo systemctl restart nordabiznes

# Verify deployment
curl -I http://localhost:5000/health

Git Remote Configuration:

# Production git config (on NORDABIZ-01)
git remote -v
# inpi    https://10.22.68.180:3000/maciejpi/nordabiz.git (fetch)
# inpi    https://10.22.68.180:3000/maciejpi/nordabiz.git (push)

Network Topology

Network Segments

Segment CIDR/Address Purpose Security Level
Public Internet 0.0.0.0/0 External user access Untrusted
WAN (Fortigate) 85.237.177.83/32 Public gateway Perimeter
INPI LAN 10.22.68.0/24 Internal services network Trusted
Localhost 127.0.0.1/8 Server-local services Isolated

DNS Configuration

External DNS (OVH):

Type Name Value TTL Purpose
A nordabiznes.pl 85.237.177.83 3600 Main domain
A www.nordabiznes.pl 85.237.177.83 3600 WWW subdomain

Internal DNS (INPI):

Type Name Value Purpose
A nordabiznes.inpi.local 10.22.68.249 Internal access
A nordabiz-01.inpi.local 10.22.68.249 Server hostname
A revproxy-01.inpi.local 10.22.68.250 Reverse proxy
A git.inpi.local 10.22.68.180 Git server

Port Mappings

NORDABIZ-01 (10.22.68.249) Port Matrix

Port Protocol Service Binding Access Purpose Security Notes
22 TCP SSH 0.0.0.0 Internal only Server administration Key-based auth
80 TCP Nginx (system) 0.0.0.0 Internal only HTTP→HTTPS redirect ⚠️ Not for NPM!
443 TCP Nginx (system) 0.0.0.0 Internal only HTTPS redirect ⚠️ Not for NPM!
5000 TCP Gunicorn/Flask 0.0.0.0 Internal only Main Application ✓ NPM uses this
5432 TCP PostgreSQL 127.0.0.1 Localhost only Database No external access
5433 TCP - - Unused Reserved for dev Docker -

⚠️ PORT 5000 - CRITICAL NOTES:

  • This is the ONLY correct port for NPM to connect to
  • Ports 80/443 are for nginx system service (redirects only)
  • Using port 80 in NPM causes infinite redirect loop
  • Always verify after NPM configuration changes

R11-REVPROXY-01 (10.22.68.250) Port Matrix

Port Protocol Service Binding Access Purpose Security Notes
22 TCP SSH 0.0.0.0 Internal only Server administration Key-based auth
80 TCP NPM 0.0.0.0 Public (via NAT) HTTP→HTTPS redirect Auto-redirect
81 TCP NPM Admin UI 0.0.0.0 Internal only NPM management Auth required
443 TCP NPM 0.0.0.0 Public (via NAT) HTTPS traffic SSL termination

r11-git-inpi (10.22.68.180) Port Matrix

Port Protocol Service Binding Access Purpose Security Notes
22 TCP SSH 0.0.0.0 Internal only Server administration Key-based auth
3000 TCP Gitea (HTTPS) 0.0.0.0 Internal only Git repository hosting Self-signed SSL

Fortigate Firewall NAT Rules

External Port Protocol Internal IP Internal Port Purpose Traffic
443 TCP 10.22.68.250 443 HTTPS public access Incoming
80 TCP 10.22.68.250 80 HTTP redirect Incoming

Firewall Rules:

  • Allow: Public → 10.22.68.250:443 (HTTPS)
  • Allow: Public → 10.22.68.250:80 (HTTP, redirects to HTTPS)
  • Deny: All other inbound traffic
  • Allow: Internal → External (outbound API calls)

Network Flow Diagrams

Successful Production Request Flow

┌─────────────────────────────────────────────────────────────────┐
│ 1. USER BROWSER                                                 │
│    https://nordabiznes.pl                                       │
└────────────────────────────┬────────────────────────────────────┘
                             │ DNS: nordabiznes.pl → 85.237.177.83
                             │ HTTPS :443
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 2. FORTIGATE FIREWALL (85.237.177.83)                          │
│    NAT: 443 → 10.22.68.250:443                                 │
└────────────────────────────┬────────────────────────────────────┘
                             │ Forward to proxy
                             │ HTTPS :443
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 3. NPM @ R11-REVPROXY-01 (10.22.68.250:443)                    │
│    • Accept HTTPS connection                                    │
│    • TLS handshake (Let's Encrypt certificate)                 │
│    • Terminate SSL/TLS                                          │
│    • Add security headers (HSTS, etc.)                          │
│    • Proxy pass to backend                                      │
└────────────────────────────┬────────────────────────────────────┘
                             │ ⚠️ CRITICAL PATH
                             │ http://10.22.68.249:5000
                             │ (NOT port 80!)
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 4. GUNICORN @ NORDABIZ-01 (10.22.68.249:5000)                  │
│    • Receive HTTP request (decrypted)                           │
│    • Load balance across 4 workers                              │
│    • Pass to Flask application                                  │
└────────────────────────────┬────────────────────────────────────┘
                             │ Process request
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 5. FLASK APP (app.py)                                           │
│    • Rate limiting check (Flask-Limiter)                        │
│    • Session validation (Flask-Login)                           │
│    • CSRF protection (Flask-WTF)                                │
│    • Route matching (90+ routes)                                │
│    • Business logic execution                                   │
└────────────────────────────┬────────────────────────────────────┘
                             │ Database query
                             │ localhost:5432
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 6. POSTGRESQL @ localhost:5432                                  │
│    • Execute SQL query (SQLAlchemy ORM)                         │
│    • Apply constraints and indexes                              │
│    • Return result set                                          │
└────────────────────────────┬────────────────────────────────────┘
                             │ Results
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ 7. FLASK APP (render response)                                  │
│    • Jinja2 template rendering                                  │
│    • JSON serialization (API routes)                            │
│    • Apply response headers                                     │
└────────────────────────────┬────────────────────────────────────┘
                             │ HTTP response
                             ▼
GUNICORN → NPM (encrypt with TLS) → FORTIGATE → USER BROWSER

Timing Breakdown:

  • DNS resolution: ~50ms
  • SSL handshake: ~100ms
  • NPM proxy: ~10ms
  • Flask processing: ~50-500ms (depends on query complexity)
  • Database query: ~10-100ms
  • Template rendering: ~20-50ms
  • Total: ~240-810ms (typical range)

Failed Request Flow (Port 80 Misconfiguration)

⚠️ This caused the 2026-01-02 production incident

USER BROWSER
    │
    │ https://nordabiznes.pl
    ▼
FORTIGATE (NAT)
    │
    ▼
NPM @ 10.22.68.250:443
    │ SSL termination
    │ ❌ WRONG: Proxy to http://10.22.68.249:80
    ▼
NGINX (System) @ 10.22.68.249:80
    │
    │ ❌ Return: 301 → https://nordabiznes.pl
    ▼
NPM (receives redirect)
    │ SSL termination again
    │ ❌ Proxy to http://10.22.68.249:80 (LOOP!)
    ▼
... Infinite redirect loop ...
    │
    ▼
BROWSER ERROR: ERR_TOO_MANY_REDIRECTS

Root Cause: NPM forwarding to port 80 instead of port 5000

Fix: Change NPM forward_port from 80 to 5000

Prevention: Always verify forward_port = 5000 after NPM changes


SSL/TLS Configuration

Let's Encrypt Certificate

Certificate Details:

  • Provider: Let's Encrypt
  • Managed By: NPM (Nginx Proxy Manager)
  • Certificate ID: 27
  • Domains:
    • nordabiznes.pl
    • www.nordabiznes.pl
  • Key Type: RSA 2048-bit
  • Validity: 90 days (auto-renewed)
  • Renewal: Automatic (30 days before expiry)

TLS Configuration:

  • Protocols: TLS 1.2, TLS 1.3 (TLS 1.0/1.1 disabled)
  • Cipher Suites: Modern ciphers only
  • HTTP/2: Enabled
  • HSTS: Enabled (max-age=31536000, includeSubDomains)

Security Headers (Added by NPM):

Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block

Certificate Verification:

# Check certificate details
openssl s_client -connect nordabiznes.pl:443 -servername nordabiznes.pl < /dev/null 2>/dev/null | \
  openssl x509 -noout -dates -subject -issuer

# Check expiry date
curl -vI https://nordabiznes.pl 2>&1 | grep -E "(expire|issuer)"

# Test SSL configuration
curl -I https://nordabiznes.pl/health
# Expected: HTTP/2 200 OK

External API Connectivity

Outbound HTTPS Connections

API Endpoint Port Authentication Purpose Rate Limit
Google Gemini AI generativelanguage.googleapis.com 443 API Key AI chat, image analysis Cost-limited
Brave Search api.search.brave.com 443 API Key News monitoring, social discovery 2000/month
Google PageSpeed googleapis.com/pagespeedonline 443 API Key SEO/performance audits 25,000/day
Google Places maps.googleapis.com/maps/api 443 API Key Business profiles, reviews Quota-based
KRS Open API api-krs.ms.gov.pl 443 Public Company verification Unlimited
Microsoft Graph graph.microsoft.com 443 OAuth 2.0 Email notifications Tenant-based

Firewall Requirements:

  • Outbound HTTPS (443): Must be allowed for all external API calls
  • DNS Resolution: Must be allowed (port 53)
  • NTP: Recommended for time sync (port 123)

API Key Storage:

  • Location: /var/www/nordabiznes/.env (production)
  • Permissions: chmod 600 (owner read/write only)
  • Owner: www-data:www-data
  • ⚠️ NEVER commit to git! (in .gitignore)

Deployment Workflow

Git-Based Deployment

┌────────────────────────────────────────────────────────────────┐
│                     DEVELOPMENT (Mac)                          │
│                                                                │
│  • Code changes                                                │
│  • Local testing (localhost:5000)                              │
│  • Git commit                                                  │
└───────────────────────────┬────────────────────────────────────┘
                            │
                            │ git push origin master
                            │ git push inpi master
                            ▼
┌────────────────────────────────────────────────────────────────┐
│                   GIT REPOSITORIES                             │
│                                                                │
│  ┌─────────────────────────────────────────────────────┐      │
│  │  GITEA @ r11-git-inpi (10.22.68.180:3000)           │      │
│  │  Repository: maciejpi/nordabiz                      │      │
│  │  Purpose: Production deployment source             │      │
│  └─────────────────────────────────────────────────────┘      │
│                                                                │
│  ┌─────────────────────────────────────────────────────┐      │
│  │  GITHUB @ github.com                                │      │
│  │  Repository: pienczyn/nordabiz                      │      │
│  │  Purpose: Cloud backup, collaboration              │      │
│  └─────────────────────────────────────────────────────┘      │
└───────────────────────────┬────────────────────────────────────┘
                            │
                            │ ssh + git pull
                            ▼
┌────────────────────────────────────────────────────────────────┐
│                 PRODUCTION (NORDABIZ-01)                       │
│                                                                │
│  1. SSH to server:                                             │
│     ssh maciejpi@10.22.68.249                                  │
│                                                                │
│  2. Pull latest code:                                          │
│     cd /var/www/nordabiznes                                    │
│     sudo -u www-data git -c http.sslVerify=false pull         │
│                                                                │
│  3. Restart service:                                           │
│     sudo systemctl restart nordabiznes                         │
│                                                                │
│  4. Verify deployment:                                         │
│     curl -I http://localhost:5000/health                       │
│     curl -I https://nordabiznes.pl/health                      │
└────────────────────────────────────────────────────────────────┘

Deployment Checklist:

  1. Code tested locally
  2. Git commit with descriptive message
  3. Push to both remotes (Gitea + GitHub)
  4. SSH to production server as maciejpi
  5. Pull latest code as www-data user
  6. Restart nordabiznes service
  7. Verify health endpoint (localhost:5000)
  8. Verify public endpoint (nordabiznes.pl)
  9. Check logs for errors (journalctl -u nordabiznes)
  10. Update release notes in app.py

Monitoring and Health Checks

Health Check Endpoint

Endpoint: GET /health Authentication: None (public) Purpose: Service availability monitoring

Response (Healthy):

HTTP/2 200 OK
Content-Type: application/json

{
  "status": "healthy",
  "timestamp": "2026-01-10T12:00:00Z",
  "database": "connected",
  "version": "1.0"
}

Health Check Commands:

# External check (via NPM)
curl -I https://nordabiznes.pl/health
# Expected: HTTP/2 200 OK

# Internal check (direct to Flask)
curl -I http://10.22.68.249:5000/health
# Expected: HTTP/1.1 200 OK

# Localhost check (from NORDABIZ-01)
curl -I http://localhost:5000/health
# Expected: HTTP/1.1 200 OK

Service Status Checks

Application Service:

# Check systemd service
sudo systemctl status nordabiznes
# Active: active (running)

# Check process
ps aux | grep gunicorn
# Should show 5 processes (1 master + 4 workers)

# Check listening ports
ss -tlnp | grep :5000
# LISTEN 0 128 0.0.0.0:5000 0.0.0.0:*

Database Service:

# Check PostgreSQL service
sudo systemctl status postgresql
# Active: active (running)

# Check database connectivity
sudo -u postgres psql -c "SELECT version();"

# Check active connections
sudo -u postgres psql -c "SELECT count(*) FROM pg_stat_activity WHERE datname='nordabiz';"

NPM Service:

# Check Docker container
docker ps | grep nginx-proxy-manager
# Should show container running

# Check NPM logs
docker logs nginx-proxy-manager_app_1 --tail 50

# Verify proxy configuration
docker exec nginx-proxy-manager_app_1 \
  sqlite3 /data/database.sqlite \
  "SELECT forward_port FROM proxy_host WHERE id = 27;"
# Expected: 5000

Planned Monitoring (Not Yet Implemented)

Zabbix Monitoring (Planned):

  • HTTPS endpoint monitoring (5-minute intervals)
  • Response time tracking
  • SSL certificate expiry alerts (30-day warning)
  • Service availability alerts
  • Database connection monitoring
  • Disk space monitoring

Backup and Disaster Recovery

PostgreSQL Database Backups

Current Status: Manual backups only Backup Location: /backup/nordabiz/ (on NORDABIZ-01)

Manual Backup Procedure:

# Create backup
sudo -u postgres pg_dump nordabiz | gzip > \
  /backup/nordabiz/nordabiz_$(date +%Y%m%d_%H%M%S).sql.gz

# Verify backup
gunzip -c /backup/nordabiz/nordabiz_20260110_120000.sql.gz | head -n 20

Restore Procedure:

# Stop application
sudo systemctl stop nordabiznes

# Drop existing database (CAUTION!)
sudo -u postgres psql -c "DROP DATABASE nordabiz;"
sudo -u postgres psql -c "CREATE DATABASE nordabiz OWNER nordabiz_app;"

# Restore from backup
gunzip -c /backup/nordabiz/nordabiz_20260110_120000.sql.gz | \
  sudo -u postgres psql nordabiz

# Grant permissions
sudo -u postgres psql -d nordabiz -c "GRANT ALL ON ALL TABLES IN SCHEMA public TO nordabiz_app;"
sudo -u postgres psql -d nordabiz -c "GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO nordabiz_app;"

# Restart application
sudo systemctl start nordabiznes

# Verify
curl -I http://localhost:5000/health

Planned: Automated daily backups via cron


VM Snapshots (Proxmox)

Snapshot Schedule:

  • Daily: 7-day retention
  • Weekly: 4-week retention
  • Monthly: 6-month retention

Snapshot Storage: Proxmox Backup Server

Restore Procedure:

  1. Access Proxmox VE web interface
  2. Navigate to VM (249 or 119)
  3. Select "Snapshots" tab
  4. Choose restore point
  5. Confirm snapshot restoration
  6. Start VM after restore
  7. Verify services are running

Recovery Time Objective (RTO): ~15 minutes Recovery Point Objective (RPO): ~24 hours (daily snapshots)


NPM Configuration Backup

Database Location: /data/database.sqlite (inside NPM container)

Manual Backup:

# Backup NPM database
docker exec nginx-proxy-manager_app_1 cat /data/database.sqlite > \
  /backup/npm/npm_$(date +%Y%m%d).sqlite

# Backup SSL certificates
docker exec nginx-proxy-manager_app_1 tar czf - /etc/letsencrypt > \
  /backup/npm/letsencrypt_$(date +%Y%m%d).tar.gz

Restore Procedure:

# Stop NPM container
docker stop nginx-proxy-manager_app_1

# Restore database
cat /backup/npm/npm_20260110.sqlite | \
  docker exec -i nginx-proxy-manager_app_1 sh -c 'cat > /data/database.sqlite'

# Restore SSL certificates
cat /backup/npm/letsencrypt_20260110.tar.gz | \
  docker exec -i nginx-proxy-manager_app_1 sh -c 'tar xzf - -C /'

# Start NPM container
docker start nginx-proxy-manager_app_1

# Verify
curl -I https://nordabiznes.pl/health

Security Configuration

Firewall Rules (Fortigate)

Inbound Rules:

Priority 1: ALLOW Public (0.0.0.0/0) → 10.22.68.250:443 (HTTPS)
Priority 2: ALLOW Public (0.0.0.0/0) → 10.22.68.250:80 (HTTP redirect)
Priority 100: DENY All other inbound traffic

Outbound Rules:

Priority 1: ALLOW 10.22.68.0/24 → Internet (HTTPS :443)
Priority 2: ALLOW 10.22.68.0/24 → Internet (DNS :53)
Priority 3: ALLOW 10.22.68.0/24 → Internet (NTP :123)
Priority 100: DENY All other outbound traffic

SSH Access Control

Allowed Users:

  • maciejpi - Administrator account (sudo access)

Authentication:

  • SSH key-based authentication (required)
  • Password authentication disabled

Firewall:

  • SSH accessible from internal network only (10.22.68.0/24)
  • No public SSH access

SSH Configuration:

# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers maciejpi

Database Security

PostgreSQL Access Control:

# postgresql.conf
listen_addresses = 'localhost'  # NO external connections!
port = 5432

# pg_hba.conf
local   nordabiz   nordabiz_app   md5       # Local socket
host    nordabiz   nordabiz_app   127.0.0.1/32   md5   # Localhost only

Connection Restrictions:

  • Localhost (127.0.0.1): Allowed
  • Internal network (10.22.68.0/24): Denied
  • External network: Denied

User Privileges:

-- nordabiz_app user (application)
GRANT CONNECT ON DATABASE nordabiz TO nordabiz_app;
GRANT USAGE ON SCHEMA public TO nordabiz_app;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO nordabiz_app;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO nordabiz_app;

-- postgres user (admin)
-- Full privileges on all databases

Troubleshooting Guide

Common Issues and Solutions

Issue 1: ERR_TOO_MANY_REDIRECTS

Symptoms:

  • Browser shows "ERR_TOO_MANY_REDIRECTS"
  • Site inaccessible via https://nordabiznes.pl
  • Internal access (localhost:5000) works fine

Cause: NPM forwarding to port 80 instead of port 5000

Solution:

# 1. Verify NPM configuration
ssh maciejpi@10.22.68.250
docker exec nginx-proxy-manager_app_1 \
  sqlite3 /data/database.sqlite \
  "SELECT id, forward_host, forward_port FROM proxy_host WHERE id = 27;"

# 2. If forward_port != 5000, fix it via NPM Web UI:
# - Access http://10.22.68.250:81
# - Edit Proxy Host #27
# - Set Forward Port to 5000
# - Save

# 3. Verify fix
curl -I https://nordabiznes.pl/health
# Expected: HTTP/2 200 OK

Prevention: Always verify forward_port = 5000 after NPM changes


Issue 2: 502 Bad Gateway

Symptoms:

  • NPM returns 502 Bad Gateway
  • External access fails
  • Internal access may or may not work

Possible Causes:

  1. Flask/Gunicorn service down
  2. Port 5000 not listening
  3. Network connectivity issue

Diagnosis:

# 1. Check Gunicorn service
ssh maciejpi@10.22.68.249
sudo systemctl status nordabiznes
# If not running: sudo systemctl start nordabiznes

# 2. Check port 5000 is listening
ss -tlnp | grep :5000
# Expected: LISTEN 0.0.0.0:5000

# 3. Test internal access
curl -I http://localhost:5000/health
# Expected: HTTP/1.1 200 OK

# 4. Test from NPM server
ssh maciejpi@10.22.68.250
curl -I http://10.22.68.249:5000/health
# Expected: HTTP/1.1 200 OK

Solution:

# Restart Gunicorn service
sudo systemctl restart nordabiznes

# Check logs for errors
sudo journalctl -u nordabiznes -n 50

Issue 3: Database Connection Errors

Symptoms:

  • Application shows "Database connection error"
  • Health endpoint returns error
  • Logs show "could not connect to server"

Diagnosis:

# 1. Check PostgreSQL service
sudo systemctl status postgresql
# If not running: sudo systemctl start postgresql

# 2. Test database connection
sudo -u postgres psql -c "SELECT 1;"
# Expected: 1

# 3. Check application database access
sudo -u www-data psql -h 127.0.0.1 -U nordabiz_app -d nordabiz -c "SELECT 1;"
# Expected: 1 (if password prompt, check .env)

# 4. Check active connections
sudo -u postgres psql -c "SELECT count(*) FROM pg_stat_activity WHERE datname='nordabiz';"

Solution:

# Restart PostgreSQL
sudo systemctl restart postgresql

# Verify connection string in .env
cat /var/www/nordabiznes/.env | grep DATABASE_URL
# Expected: postgresql://nordabiz_app:PASSWORD@localhost:5432/nordabiz

Issue 4: SSL Certificate Expired

Symptoms:

  • Browser shows "Your connection is not private"
  • SSL certificate warning
  • Cert expired error

Diagnosis:

# Check certificate expiry
openssl s_client -connect nordabiznes.pl:443 -servername nordabiznes.pl < /dev/null 2>/dev/null | \
  openssl x509 -noout -dates

# Check NPM logs for renewal errors
docker logs nginx-proxy-manager_app_1 | grep -i letsencrypt

Solution:

# Manual certificate renewal via NPM
# 1. Access NPM Web UI: http://10.22.68.250:81
# 2. Go to SSL Certificates
# 3. Find certificate ID 27
# 4. Click "Renew" button

# OR restart NPM to trigger auto-renewal
docker restart nginx-proxy-manager_app_1

Architecture Documentation

Data Flow Documentation

Incident Documentation

Infrastructure Documentation


Maintenance Notes

When to Update This Diagram

✏️ UPDATE when:

  • New server added to infrastructure
  • IP address changes
  • Port configuration changes
  • Network topology changes (new firewall rules, NAT changes)
  • SSL/TLS configuration updates
  • Deployment procedure changes
  • Critical configuration changes (like NPM proxy settings)

DON'T UPDATE for:

  • Application code changes (not infrastructure)
  • Database schema changes (covered in database diagram)
  • New Flask routes (covered in component diagram)
  • Dependency updates (same infrastructure)

Review Frequency

  • After incidents: Immediate review and update
  • After infrastructure changes: Within 24 hours
  • Quarterly: Verify accuracy against actual configuration
  • Before major deployments: Review deployment section

Verification Checklist

  • All IP addresses are current
  • All port numbers are correct
  • NPM proxy configuration verified (port 5000!)
  • DNS records match actual configuration
  • SSL certificate status checked
  • Firewall rules documented accurately
  • Service status verified on all servers
  • Health check endpoints tested
  • Backup procedures tested

Glossary

Term Definition
NPM Nginx Proxy Manager - Docker-based reverse proxy with web UI
NAT Network Address Translation - maps public IP to internal IPs
SSL Termination Decrypting HTTPS at proxy, forwarding HTTP to backend
Fortigate Enterprise firewall/router device
Gunicorn Python WSGI HTTP server for running Flask applications
Let's Encrypt Free automated SSL/TLS certificate authority
HSTS HTTP Strict Transport Security - forces HTTPS
Proxmox VE Virtualization platform for managing VMs
Systemd Linux init system and service manager
WSGI Web Server Gateway Interface - Python web server standard
VM Virtual Machine
vCPU Virtual CPU core
ORM Object-Relational Mapping (SQLAlchemy)

Document Status: Complete Diagram Validated: 2026-01-10 Production Verified: 2026-01-10 Mermaid Syntax: v10.6+ Renders in: GitHub, GitLab, VS Code (with Mermaid extension)


⚠️ CRITICAL CONFIGURATION REMINDER:

NPM Proxy Host 27 MUST use:

  • Forward Host: 10.22.68.249
  • Forward Port: 5000 (NOT 80!)

Always verify after changes:

curl -I https://nordabiznes.pl/health
# Expected: HTTP/2 200 OK

See: docs/INCIDENT_REPORT_20260102.md for details on port 80 incident