ASF-Nabd/ASF_tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init tools repo

Browse Source
This commit is contained in:
mahmamdouh
2025-11-23 19:57:05 +01:00
commit d778206940
35 changed files with 6197 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

649
asf-cloud-server/TBM_devbench/docs/API.md Normal file
View File

@@ -0,0 +1,649 @@
# DevBench Manager - API Documentation
## Base URL
```
http://localhost:9090
https://tbm.nabd-co.com
```
## Authentication
All API endpoints (except `/login` and `/health`) require authentication via session cookies.
### Session Cookie
- **Name**: `connect.sid`
- **Type**: HTTP-only
- **Secure**: true (in production with HTTPS)
- **SameSite**: Lax
## Endpoints
### Health Check
#### GET /health
Check application health status.
**Authentication**: Not required
**Response**:
```json
{
"status": "ok",
"timestamp": "2025-11-23T12:00:00.000Z",
"version": "1.0.0"
}
```
**Status Codes**:
- `200`: Application is healthy
---
### Authentication
#### GET /login
Display login page.
**Authentication**: Not required
**Response**: HTML page
---
#### POST /login
Authenticate user and create session.
**Authentication**: Not required
**Request Body**:
```json
{
"username": "admin",
"password": "admin123"
}
```
**Response**: Redirect to `/dashboard` or `/admin`
**Status Codes**:
- `302`: Success, redirect to dashboard
- `200`: Failed, re-render login with error
---
#### GET /logout
Destroy session and logout user.
**Authentication**: Required
**Response**: Redirect to `/login`
**Status Codes**:
- `302`: Success, redirect to login
---
### User Dashboard
#### GET /dashboard
Display user's DevBench dashboard.
**Authentication**: Required (User role)
**Response**: HTML page with user's DevBenches
**Status Codes**:
- `200`: Success
- `302`: Redirect to `/admin` if user is admin
- `302`: Redirect to `/login` if not authenticated
---
#### GET /help
Display help page with SSH configuration guide.
**Authentication**: Required
**Response**: HTML page with documentation
**Status Codes**:
- `200`: Success
- `302`: Redirect to `/login` if not authenticated
---
### DevBench Management
#### POST /create-devbench
Create a new DevBench.
**Authentication**: Required (User role)
**Request Body**:
```json
{
"name": "test-vm"
}
```
**Validation**:
- Name must match pattern: `[a-zA-Z0-9_-]+`
- Name must be unique for user
**Response**:
```json
{
"success": true,
"devbenchId": 123
}
```
**Error Response**:
```json
{
"error": "DevBench with this name already exists"
}
```
**Status Codes**:
- `200`: Success
- `400`: Validation error
- `500`: Database error
**Side Effects**:
- Creates database record with status "creating"
- Spawns provision script execution
- Sends real-time updates via WebSocket
---
#### POST /delete-devbench/:id
Delete a DevBench.
**Authentication**: Required (User role, owner only)
**Parameters**:
- `id`: DevBench ID (integer)
**Response**:
```json
{
"success": true
}
```
**Error Response**:
```json
{
"error": "DevBench not found"
}
```
**Status Codes**:
- `200`: Success
- `404`: DevBench not found
- `500`: Database error
**Side Effects**:
- Executes delete command on remote host
- Removes database record
---
#### POST /activate-devbench/:id
Activate a DevBench.
**Authentication**: Required (User role, owner only)
**Parameters**:
- `id`: DevBench ID (integer)
**Response**:
```json
{
"success": true
}
```
**Error Response**:
```json
{
"error": "DevBench not found"
}
```
**Status Codes**:
- `200`: Success
- `404`: DevBench not found
**Side Effects**:
- Executes activate command on remote host
- Updates status via WebSocket
---
#### GET /check-status/:id
Check DevBench status.
**Authentication**: Required (User role, owner only)
**Parameters**:
- `id`: DevBench ID (integer)
**Response**:
```json
{
"status": "active"
}
```
**Possible Status Values**:
- `active`: VM is running
- `inactive`: VM is stopped
- `creating`: VM is being created
- `error`: VM creation failed
- `unknown`: Status cannot be determined
**Status Codes**:
- `200`: Success
**Side Effects**:
- Updates database with current status
---
### Admin Panel
#### GET /admin
Display admin dashboard.
**Authentication**: Required (Admin role)
**Response**: HTML page with users and DevBenches
**Status Codes**:
- `200`: Success
- `403`: Access denied (not admin)
- `302`: Redirect to `/login` if not authenticated
---
#### POST /admin/add-user
Create a new user.
**Authentication**: Required (Admin role)
**Request Body**:
```json
{
"username": "john",
"email": "john@example.com",
"password": "password123"
}
```
**Validation**:
- Username must match pattern: `[a-zA-Z]+` (letters only)
- Username must be unique
- Email must be valid format
- Password required
**Response**:
```json
{
"success": true
}
```
**Error Response**:
```json
{
"error": "Username already exists"
}
```
**Status Codes**:
- `200`: Success
- `400`: Validation error
- `500`: Database error
---
#### POST /admin/delete-user/:id
Delete a user and all their DevBenches.
**Authentication**: Required (Admin role)
**Parameters**:
- `id`: User ID (integer)
**Response**:
```json
{
"success": true
}
```
**Error Response**:
```json
{
"error": "Database error"
}
```
**Status Codes**:
- `200`: Success
- `500`: Database error
**Side Effects**:
- Deletes all user's DevBenches
- Deletes user record
- Cannot delete admin users
---
#### POST /admin/reset-password/:id
Reset user password.
**Authentication**: Required (Admin role)
**Parameters**:
- `id`: User ID (integer)
**Request Body**:
```json
{
"newPassword": "newpassword123"
}
```
**Response**:
```json
{
"success": true
}
```
**Error Response**:
```json
{
"error": "Database error"
}
```
**Status Codes**:
- `200`: Success
- `500`: Database error
**Side Effects**:
- Updates user password (hashed with bcrypt)
- Cannot reset admin passwords
---
### User Info
#### GET /api/user-info
Get current user information for WebSocket registration.
**Authentication**: Required
**Response**:
```json
{
"userId": 1,
"username": "admin",
"isAdmin": true
}
```
**Status Codes**:
- `200`: Success
- `302`: Redirect to `/login` if not authenticated
---
## WebSocket API
### Connection
```javascript
const ws = new WebSocket('ws://localhost:9090');
```
### Registration
After connection, register with user ID:
```javascript
ws.send(JSON.stringify({
type: 'register',
userId: 1
}));
```
### Message Types
#### script_output
Real-time output from provision script.
```json
{
"type": "script_output",
"devbenchId": 123,
"data": "Cloning VM...\n"
}
```
#### script_complete
Script execution completed.
```json
{
"type": "script_complete",
"devbenchId": 123,
"exitCode": 0
}
```
#### status_update
DevBench status changed.
```json
{
"type": "status_update",
"devbenchId": 123,
"status": "active"
}
```
---
## Error Responses
### Standard Error Format
```json
{
"error": "Error message description"
}
```
### Common HTTP Status Codes
- `200`: Success
- `302`: Redirect
- `400`: Bad Request (validation error)
- `401`: Unauthorized (not authenticated)
- `403`: Forbidden (insufficient permissions)
- `404`: Not Found
- `500`: Internal Server Error
---
## Rate Limiting
Currently not implemented. Consider adding rate limiting for production:
- Login attempts: 5 per minute
- DevBench creation: 10 per hour per user
- API calls: 100 per minute per user
---
## Data Models
### User
```json
{
"id": 1,
"username": "admin",
"email": "admin@example.com",
"password": "$2a$10$...", // bcrypt hash
"is_admin": 1,
"created_at": "2025-11-23T12:00:00.000Z"
}
```
### DevBench
```json
{
"id": 1,
"user_id": 1,
"name": "test-vm",
"actual_name": "admin_test-vm",
"status": "active",
"ssh_info": "6004",
"vnc_info": "5004",
"created_at": "2025-11-23T12:00:00.000Z",
"updated_at": "2025-11-23T12:05:00.000Z"
}
```
---
## Examples
### Create DevBench with cURL
```bash
# Login first to get session cookie
curl -c cookies.txt -X POST http://localhost:9090/login \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "username=admin&password=admin123"
# Create DevBench
curl -b cookies.txt -X POST http://localhost:9090/create-devbench \
-H "Content-Type: application/json" \
-d '{"name":"test-vm"}'
```
### Check Status with cURL
```bash
curl -b cookies.txt http://localhost:9090/check-status/1
```
### Add User with cURL
```bash
curl -b cookies.txt -X POST http://localhost:9090/admin/add-user \
-H "Content-Type: application/json" \
-d '{
"username":"john",
"email":"john@example.com",
"password":"password123"
}'
```
### WebSocket with JavaScript
```javascript
// Connect
const ws = new WebSocket('ws://localhost:9090');
// Register
ws.onopen = () => {
fetch('/api/user-info')
.then(res => res.json())
.then(data => {
ws.send(JSON.stringify({
type: 'register',
userId: data.userId
}));
});
};
// Listen for messages
ws.onmessage = (event) => {
const message = JSON.parse(event.data);
console.log('Received:', message);
if (message.type === 'script_output') {
// Update UI with script output
document.getElementById('log').textContent += message.data;
}
};
```
---
## Security Considerations
### Authentication
- Session-based authentication
- HTTP-only cookies
- Password hashing with bcrypt (10 rounds)
### Authorization
- Role-based access control (Admin/User)
- Owner-only access to DevBenches
- Admin-only routes protected
### Input Validation
- Username: Letters only
- DevBench name: Alphanumeric, hyphens, underscores
- SQL injection prevention: Parameterized queries
- XSS prevention: EJS auto-escaping
### Best Practices
- Always use HTTPS in production
- Change default admin password
- Use strong session secret
- Implement rate limiting
- Add CSRF protection
- Enable security headers
---
## Changelog
### Version 1.0.0 (2025-11-23)
- Initial API release
- User authentication
- DevBench management
- Admin panel
- WebSocket support
- Help page
- Dark theme support

400
asf-cloud-server/TBM_devbench/docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,400 @@
# DevBench Manager - Architecture Documentation
## System Overview
DevBench Manager is a web-based application for managing virtual machine (VM) development environments. It provides a centralized interface for users to create, manage, and access their DevBench VMs through a secure web portal.
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────┐
│ User Browser │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Dashboard │ │ Admin Panel │ │ Help Page │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└────────────────────────┬────────────────────────────────────────┘
│ HTTPS/WSS
▼
┌─────────────────────────────────────────────────────────────────┐
│ Caddy Reverse Proxy │
│ (tbm.nabd-co.com) │
└────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ DevBench Manager Container │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Express.js Server │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ │
│ │ │ Routes │ │ WebSocket │ │ Auth │ │ │
│ │ └────────────┘ └────────────┘ └────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ SQLite Database │ │
│ │ ┌────────────┐ ┌────────────┐ │ │
│ │ │ Users │ │ DevBenches │ │ │
│ │ └────────────┘ └────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Provision Script Executor │ │
│ │ (provision_vm.sh) │ │
│ └──────────────────────────────────────────────────────────┘ │
└────────────────────────┬────────────────────────────────────────┘
│ SSH (Port 49152)
▼
┌─────────────────────────────────────────────────────────────────┐
│ Remote VM Host Server │
│ (asf-server.duckdns.org) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ VirtualBox Manager │ │
│ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │
│ │ │ VM 1 │ │ VM 2 │ │ VM 3 │ │ VM N │ │ │
│ │ │ SSH: │ │ SSH: │ │ SSH: │ │ SSH: │ │ │
│ │ │ 6001 │ │ 6002 │ │ 6003 │ │ 600N │ │ │
│ │ │ VNC: │ │ VNC: │ │ VNC: │ │ VNC: │ │ │
│ │ │ 5001 │ │ 5002 │ │ 5003 │ │ 500N │ │ │
│ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
## Component Architecture
### 1. Frontend Layer
#### Technologies
- **EJS Templates**: Server-side rendering
- **Bootstrap 5**: UI framework
- **Font Awesome**: Icons
- **Custom CSS**: Theming (light/dark mode)
- **Vanilla JavaScript**: Client-side interactivity
#### Components
- **Dashboard**: User's DevBench management interface
- **Admin Panel**: User and system management
- **Help Page**: Documentation and SSH config guide
- **Login Page**: Authentication interface
### 2. Backend Layer
#### Technologies
- **Node.js**: Runtime environment
- **Express.js**: Web framework
- **WebSocket (ws)**: Real-time communication
- **SQLite3**: Database
- **bcryptjs**: Password hashing
- **express-session**: Session management
#### Core Modules
##### Authentication Module
```javascript
- Session-based authentication
- Password hashing with bcrypt
- Role-based access control (Admin/User)
- Middleware for route protection
```
##### DevBench Management Module
```javascript
- Create DevBench
- Delete DevBench
- Activate DevBench
- Status monitoring
- Real-time log streaming
```
##### WebSocket Module
```javascript
- Real-time script output
- Status updates
- User-specific broadcasting
- Connection management
```
##### Provision Script Executor
```javascript
- SSH connection to remote host
- Script execution via sshpass
- Output parsing
- Error handling
- Timeout management (30 minutes)
```
### 3. Database Layer
#### Schema
**Users Table**
```sql
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
email TEXT NOT NULL,
password TEXT NOT NULL,
is_admin INTEGER DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
```
**DevBenches Table**
```sql
CREATE TABLE devbenches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
name TEXT NOT NULL,
actual_name TEXT,
status TEXT DEFAULT 'inactive',
ssh_info TEXT,
vnc_info TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users (id)
)
```
### 4. Infrastructure Layer
#### Docker Container
```yaml
- Base Image: Node.js 18
- Exposed Port: 3001 (mapped to 9090)
- Volumes:
- ./data:/app/data (Database persistence)
- ./logs:/app/logs (Log files)
- ./provision_vm.sh:/app/provision_vm.sh (Script)
- Networks:
- app-network (Internal)
- caddy_network (External proxy)
```
#### Reverse Proxy (Caddy)
```
- Domain: tbm.nabd-co.com
- SSL/TLS: Automatic (Let's Encrypt)
- Proxy Target: devbench-manager:3001
```
## Data Flow
### DevBench Creation Flow
```
1. User clicks "Create DevBench"
↓
2. Frontend sends POST /create-devbench
↓
3. Backend validates input
↓
4. Database record created (status: creating)
↓
5. WebSocket connection established
↓
6. provision_vm.sh executed via SSH
↓
7. Real-time output streamed via WebSocket
↓
8. Script parses VM info (name, SSH port, VNC port)
↓
9. Database updated with connection info
↓
10. Frontend receives completion notification
↓
11. Page refreshes to show active DevBench
```
### Authentication Flow
```
1. User submits login form
↓
2. Backend validates credentials
↓
3. Password compared with bcrypt
↓
4. Session created on success
↓
5. User redirected to dashboard/admin
↓
6. Session cookie stored in browser
↓
7. Subsequent requests include session
↓
8. Middleware validates session
```
### Status Monitoring Flow
```
1. Periodic check (every 60 seconds)
↓
2. For each DevBench in database
↓
3. Execute provision_vm.sh status <vm_name>
↓
4. Parse output for status
↓
5. Update database if changed
↓
6. Broadcast update via WebSocket
↓
7. Frontend updates UI in real-time
```
## Security Architecture
### Authentication & Authorization
- **Password Hashing**: bcrypt with salt rounds
- **Session Management**: Secure HTTP-only cookies
- **Role-Based Access**: Admin vs User permissions
- **Route Protection**: Middleware guards
### Input Validation
- **Username**: Letters only, no special characters
- **DevBench Name**: Alphanumeric, hyphens, underscores
- **SQL Injection**: Parameterized queries
- **XSS Prevention**: EJS auto-escaping
### Network Security
- **HTTPS**: Enforced via Caddy
- **SSH**: Password-based (sshpass) with timeout
- **Container Isolation**: Docker networking
- **Port Exposure**: Minimal (only 9090)
## Scalability Considerations
### Current Limitations
- Single SQLite database (not suitable for high concurrency)
- SSH-based provisioning (sequential, not parallel)
- In-memory WebSocket connections (lost on restart)
- Single container deployment
### Future Improvements
- **Database**: Migrate to PostgreSQL/MySQL
- **Queue System**: Redis/RabbitMQ for async jobs
- **Load Balancing**: Multiple container instances
- **Persistent WebSocket**: Redis pub/sub
- **Caching**: Redis for session and status data
## Monitoring & Logging
### Application Logs
- Location: `/app/logs/`
- Format: Timestamped console output
- Rotation: Manual (not automated)
### Provision Script Logs
- Location: `/var/log/devbench/`
- Format: Timestamped with operation details
- Retention: Indefinite
### Health Checks
- Endpoint: `/health`
- Interval: 30 seconds
- Timeout: 10 seconds
- Retries: 3
## Technology Stack Summary
| Layer | Technology | Purpose |
|-------|-----------|---------|
| Frontend | EJS, Bootstrap 5, JavaScript | UI rendering and interactivity |
| Backend | Node.js, Express.js | Web server and API |
| Real-time | WebSocket (ws) | Live updates |
| Database | SQLite3 | Data persistence |
| Authentication | bcryptjs, express-session | Security |
| Container | Docker, Docker Compose | Deployment |
| Proxy | Caddy | SSL/TLS and routing |
| VM Management | Bash, SSH, VirtualBox | VM provisioning |
## Deployment Architecture
### Production Environment
```
Internet
↓
Caddy (Port 443) → SSL Termination
↓
Docker Network (caddy_network)
↓
DevBench Manager Container (Port 3001)
↓
SQLite Database (Volume: ./data)
↓
SSH Connection (Port 49152)
↓
Remote VM Host
```
### Development Environment
```
localhost:9090
↓
DevBench Manager Container
↓
Local SQLite Database
↓
SSH to Remote Host
```
## Configuration Management
### Environment Variables
- `NODE_ENV`: production/development
- `PORT`: Application port (default: 3001)
- `SECRET_KEY`: Session secret
- `ADMIN_EMAIL`: Default admin email
- `ADMIN_PASSWORD`: Default admin password
### Configuration Files
- `config.js`: Application configuration
- `docker-compose.yml`: Container orchestration
- `Dockerfile`: Container build instructions
- `.env`: Environment variables (optional)
## Error Handling
### Application Errors
- Database errors: Logged and returned as 500
- Authentication errors: Returned as 401/403
- Validation errors: Returned as 400
- Script errors: Logged and broadcast via WebSocket
### Script Errors
- Timeout: 30 minutes (1800 seconds)
- SSH failures: Logged and status set to 'error'
- Parsing failures: Status set to 'error'
- Network issues: Retry not implemented
## Performance Considerations
### Bottlenecks
1. **SSH Connection**: Sequential, blocking
2. **Script Execution**: Long-running (up to 30 min)
3. **Status Checks**: Every 60 seconds for all VMs
4. **SQLite**: Limited concurrent writes
### Optimizations
- WebSocket for real-time updates (no polling)
- Session caching in memory
- Static asset serving via Express
- Minimal database queries
## Maintenance & Operations
### Backup Strategy
- Database: Copy `./data/devbench.db`
- Logs: Archive `./logs/` directory
- Configuration: Version control
### Update Procedure
1. Pull latest code
2. Run `./deploy.sh`
3. Verify health endpoint
4. Check logs for errors
### Troubleshooting
- Check container logs: `docker-compose logs -f`
- Verify network: `docker network inspect caddy_network`
- Test SSH: `ssh -p 49152 asf@asf-server.duckdns.org`
- Check database: `sqlite3 data/devbench.db`

679
asf-cloud-server/TBM_devbench/docs/DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,679 @@
# DevBench Manager - Deployment Guide
## Table of Contents
1. [Prerequisites](#prerequisites)
2. [Quick Start](#quick-start)
3. [Detailed Deployment](#detailed-deployment)
4. [Configuration](#configuration)
5. [Verification](#verification)
6. [Troubleshooting](#troubleshooting)
7. [Maintenance](#maintenance)
8. [Backup & Recovery](#backup--recovery)
## Prerequisites
### Required Software
- **Docker**: Version 20.10 or higher
- **Docker Compose**: Version 2.0 or higher
- **Git**: For cloning the repository
- **SSH Access**: To remote VM host (asf-server.duckdns.org:49152)
### System Requirements
- **OS**: Linux (Ubuntu 20.04+ recommended), macOS, or Windows with WSL2
- **RAM**: Minimum 512MB for container
- **Disk**: Minimum 1GB free space
- **Network**: Internet connection for Docker images and SSH
### Access Requirements
- SSH credentials for VM host
- Domain name configured (optional, for Caddy)
- Port 9090 available (or configure different port)
## Quick Start
### One-Command Deployment
```bash
# Clone the repository
git clone <repository-url>
cd ASF_devbench
# Run deployment script
chmod +x deploy.sh
./deploy.sh
```
That's it! The application will be available at:
- **Direct**: http://localhost:9090
- **Via Caddy**: https://tbm.nabd-co.com (if configured)
### Default Credentials
- **Username**: admin
- **Password**: admin123
⚠️ **Important**: Change the default password after first login!
## Detailed Deployment
### Step 1: Clone Repository
```bash
git clone <repository-url>
cd ASF_devbench
```
### Step 2: Review Configuration
Check `config.js` for default settings:
```javascript
{
port: 3001,
database: { path: './data/devbench.db' },
session: { secret: 'your-secret-key' },
defaultAdmin: {
username: 'admin',
password: 'admin123',
email: 'admin@nabd-co.com'
}
}
```
### Step 3: Set Environment Variables (Optional)
Create `.env` file:
```bash
NODE_ENV=production
PORT=3001
SECRET_KEY=your-secure-secret-key
ADMIN_EMAIL=admin@yourdomain.com
ADMIN_PASSWORD=your-secure-password
```
### Step 4: Prepare Directories
```bash
# Create necessary directories
mkdir -p data logs public/downloads
# Set permissions
chmod 755 data logs
chmod +x provision_vm.sh deploy.sh
```
### Step 5: Configure Docker Network
```bash
# Create Caddy network (if using reverse proxy)
docker network create caddy_network
```
### Step 6: Deploy with Docker Compose
```bash
# Build and start container
docker-compose up -d --build
# Check status
docker-compose ps
# View logs
docker-compose logs -f
```
### Step 7: Verify Deployment
```bash
# Check health endpoint
curl http://localhost:9090/health
# Expected response:
# {"status":"ok","timestamp":"2025-11-23T...","version":"1.0.0"}
```
## Configuration
### Docker Compose Configuration
Edit `docker-compose.yml`:
```yaml
services:
devbench-manager:
ports:
- "9090:3001" # Change external port here
environment:
- NODE_ENV=production
- SECRET_KEY=${SECRET_KEY:-dev-secret-key}
volumes:
- ./data:/app/data
- ./logs:/app/logs
- ./provision_vm.sh:/app/provision_vm.sh:ro
```
### Caddy Reverse Proxy
Add to your Caddyfile:
```
tbm.nabd-co.com {
reverse_proxy devbench-manager:3001
}
```
Ensure Caddy container is on `caddy_network`:
```bash
# Check Caddy network
docker network inspect caddy_network
# Connect Caddy to network if needed
docker network connect caddy_network caddy-container-name
```
### SSH Configuration
The application connects to the VM host via SSH. Ensure:
1. **Host**: asf-server.duckdns.org
2. **Port**: 49152
3. **User**: asf
4. **Password**: ASF (configured in provision_vm.sh)
Test SSH connection:
```bash
ssh -p 49152 asf@asf-server.duckdns.org
```
### Firewall Configuration
Open required ports:
```bash
# For direct access
sudo ufw allow 9090/tcp
# For Caddy (if using)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
```
## Verification
### 1. Container Status
```bash
# Check if container is running
docker ps | grep devbench-manager
# Expected output:
# CONTAINER ID IMAGE STATUS PORTS
# abc123... devbench-manager:latest Up 2 minutes 0.0.0.0:9090->3001/tcp
```
### 2. Health Check
```bash
# Test health endpoint
curl http://localhost:9090/health
# Or with jq for formatted output
curl -s http://localhost:9090/health | jq
```
### 3. Network Connectivity
```bash
# Check networks
docker inspect devbench-manager --format='{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}'
# Expected: app-network caddy_network
```
### 4. Database
```bash
# Check if database was created
ls -la data/devbench.db
# Connect to database
docker exec -it devbench-manager sqlite3 /app/data/devbench.db
# Run query
sqlite> SELECT * FROM users;
```
### 5. Logs
```bash
# View application logs
docker-compose logs -f
# View last 50 lines
docker-compose logs --tail=50
# View specific service
docker-compose logs devbench-manager
```
### 6. Web Interface
1. Open browser: http://localhost:9090
2. Login with admin credentials
3. Create a test DevBench
4. Verify real-time log output
5. Check connection information
## Troubleshooting
### Container Won't Start
**Problem**: Container exits immediately
**Solution**:
```bash
# Check logs
docker-compose logs
# Common issues:
# 1. Port already in use
sudo lsof -i :9090
# Kill process or change port in docker-compose.yml
# 2. Permission issues
sudo chown -R $USER:$USER data logs
# 3. Missing provision script
ls -la provision_vm.sh
chmod +x provision_vm.sh
```
### Cannot Access Web Interface
**Problem**: Connection refused or timeout
**Solution**:
```bash
# 1. Check if container is running
docker ps | grep devbench-manager
# 2. Check port mapping
docker port devbench-manager
# 3. Test from inside container
docker exec devbench-manager wget -O- http://localhost:3001/health
# 4. Check firewall
sudo ufw status
sudo ufw allow 9090/tcp
# 5. Check if port is listening
sudo netstat -tlnp | grep 9090
```
### Caddy Proxy Not Working
**Problem**: Cannot access via domain
**Solution**:
```bash
# 1. Check if both containers are on caddy_network
docker network inspect caddy_network
# 2. Connect devbench-manager to network
docker network connect caddy_network devbench-manager
# 3. Restart Caddy
docker restart caddy-container-name
# 4. Check Caddy logs
docker logs caddy-container-name
# 5. Test from Caddy container
docker exec caddy-container-name wget -O- http://devbench-manager:3001/health
```
### SSH Connection Fails
**Problem**: Cannot connect to VM host
**Solution**:
```bash
# 1. Test SSH manually
ssh -p 49152 asf@asf-server.duckdns.org
# 2. Check if sshpass is installed in container
docker exec devbench-manager which sshpass
# 3. Check provision script
docker exec devbench-manager cat /app/provision_vm.sh | grep SSH_
# 4. Test from container
docker exec -it devbench-manager bash
./provision_vm.sh status test_vm
```
### Database Errors
**Problem**: Database locked or corrupted
**Solution**:
```bash
# 1. Stop container
docker-compose down
# 2. Backup database
cp data/devbench.db data/devbench.db.backup
# 3. Check database integrity
sqlite3 data/devbench.db "PRAGMA integrity_check;"
# 4. If corrupted, restore from backup or recreate
rm data/devbench.db
docker-compose up -d
```
### WebSocket Not Working
**Problem**: Real-time updates not appearing
**Solution**:
```bash
# 1. Check browser console for errors
# Open DevTools → Console
# 2. Verify WebSocket connection
# Look for "WebSocket connected" message
# 3. Check if behind proxy
# Ensure proxy supports WebSocket upgrade
# 4. Test WebSocket endpoint
wscat -c ws://localhost:9090
```
## Maintenance
### Regular Tasks
#### Daily
- Monitor logs for errors
- Check disk space
- Verify health endpoint
#### Weekly
- Review user activity
- Clean up old logs
- Check for updates
#### Monthly
- Backup database
- Review security
- Update dependencies
### Updating the Application
```bash
# 1. Pull latest changes
git pull origin main
# 2. Backup database
cp data/devbench.db data/devbench.db.$(date +%Y%m%d)
# 3. Rebuild and restart
docker-compose down
docker-compose up -d --build
# 4. Verify
curl http://localhost:9090/health
```
### Viewing Logs
```bash
# Real-time logs
docker-compose logs -f
# Last 100 lines
docker-compose logs --tail=100
# Specific time range
docker-compose logs --since 2h
# Save logs to file
docker-compose logs > logs/docker-$(date +%Y%m%d).log
```
### Cleaning Up
```bash
# Remove stopped containers
docker container prune -f
# Remove unused images
docker image prune -a -f
# Remove unused volumes
docker volume prune -f
# Clean everything
docker system prune -a -f
```
### Scaling (Future)
For multiple instances:
```bash
# Scale to 3 instances
docker-compose up -d --scale devbench-manager=3
# Use load balancer (nginx/haproxy)
# Configure session persistence
```
## Backup & Recovery
### Backup Strategy
#### Database Backup
```bash
# Manual backup
cp data/devbench.db backups/devbench-$(date +%Y%m%d-%H%M%S).db
# Automated backup script
#!/bin/bash
BACKUP_DIR="backups"
mkdir -p $BACKUP_DIR
cp data/devbench.db $BACKUP_DIR/devbench-$(date +%Y%m%d).db
# Keep only last 7 days
find $BACKUP_DIR -name "devbench-*.db" -mtime +7 -delete
```
#### Full Backup
```bash
# Backup everything
tar -czf devbench-backup-$(date +%Y%m%d).tar.gz \
data/ \
logs/ \
config.js \
.env \
docker-compose.yml
```
### Recovery
#### Restore Database
```bash
# 1. Stop container
docker-compose down
# 2. Restore database
cp backups/devbench-20251123.db data/devbench.db
# 3. Start container
docker-compose up -d
# 4. Verify
curl http://localhost:9090/health
```
#### Full Recovery
```bash
# 1. Extract backup
tar -xzf devbench-backup-20251123.tar.gz
# 2. Deploy
./deploy.sh
# 3. Verify all services
```
### Disaster Recovery
#### Complete System Failure
```bash
# 1. Fresh installation
git clone <repository-url>
cd ASF_devbench
# 2. Restore backups
cp /backup/location/devbench.db data/
cp /backup/location/.env .
# 3. Deploy
./deploy.sh
# 4. Verify users and DevBenches
```
## Security Hardening
### Change Default Credentials
```bash
# 1. Login as admin
# 2. Go to admin panel
# 3. Reset admin password
# Or use API:
curl -X POST http://localhost:9090/admin/reset-password/1 \
-H "Content-Type: application/json" \
-d '{"newPassword":"new-secure-password"}'
```
### Update Session Secret
Edit `.env`:
```bash
SECRET_KEY=$(openssl rand -base64 32)
```
Restart:
```bash
docker-compose restart
```
### Enable HTTPS
Use Caddy for automatic HTTPS:
```
tbm.nabd-co.com {
reverse_proxy devbench-manager:3001
}
```
### Restrict Access
Use firewall:
```bash
# Allow only specific IPs
sudo ufw allow from 192.168.1.0/24 to any port 9090
```
## Monitoring
### Health Monitoring
```bash
# Simple health check script
#!/bin/bash
if curl -sf http://localhost:9090/health > /dev/null; then
echo "OK"
else
echo "FAILED"
# Send alert
fi
```
### Log Monitoring
```bash
# Watch for errors
docker-compose logs -f | grep -i error
# Count errors
docker-compose logs | grep -i error | wc -l
```
### Resource Monitoring
```bash
# Container stats
docker stats devbench-manager
# Disk usage
du -sh data/ logs/
# Database size
ls -lh data/devbench.db
```
## Performance Tuning
### Database Optimization
```bash
# Vacuum database
docker exec devbench-manager sqlite3 /app/data/devbench.db "VACUUM;"
# Analyze database
docker exec devbench-manager sqlite3 /app/data/devbench.db "ANALYZE;"
```
### Container Resources
Edit `docker-compose.yml`:
```yaml
services:
devbench-manager:
deploy:
resources:
limits:
cpus: '1.0'
memory: 512M
reservations:
cpus: '0.5'
memory: 256M
```
## Support
### Getting Help
1. Check logs: `docker-compose logs`
2. Review documentation in `/docs`
3. Check GitHub issues
4. Contact system administrator
### Reporting Issues
Include:
- Docker version: `docker --version`
- Docker Compose version: `docker-compose --version`
- Container logs: `docker-compose logs`
- Error messages
- Steps to reproduce

530
asf-cloud-server/TBM_devbench/docs/STRUCTURE.md Normal file
View File

@@ -0,0 +1,530 @@
# DevBench Manager - Project Structure
## Directory Tree
```
ASF_devbench/
├── docs/ # Documentation
│ ├── ARCHITECTURE.md # System architecture
│ ├── STRUCTURE.md # This file
│ ├── DEPLOYMENT.md # Deployment guide
│ └── API.md # API documentation
│
├── public/ # Static assets
│ ├── css/
│ │ └── style.css # Custom styles + dark theme
│ ├── images/
│ │ ├── nabd-logo.svg # Company logo (light)
│ │ ├── nabd-logo-white.svg # Company logo (dark)
│ │ └── tbm-icon.png # TBM branding icon
│ └── downloads/
│ └── db_vm_ssh_config_manager.exe # SSH config tool
│
├── views/ # EJS templates
│ ├── layout.ejs # Base layout with navbar
│ ├── login.ejs # Login page
│ ├── dashboard.ejs # User dashboard
│ ├── admin.ejs # Admin panel
│ └── help.ejs # Help/documentation page
│
├── data/ # Database storage (created on deploy)
│ └── devbench.db # SQLite database
│
├── logs/ # Application logs (created on deploy)
│ └── *.log # Log files
│
├── server.js # Main application server
├── config.js # Configuration settings
├── package.json # Node.js dependencies
├── provision_vm.sh # VM provisioning script
│
├── Dockerfile # Docker image definition
├── docker-compose.yml # Container orchestration
├── .dockerignore # Docker build exclusions
│
├── deploy.sh # Deployment script
├── install.sh # Local installation script
├── start.sh # Start script (Docker/Node)
├── cleanup.sh # Cleanup script
│
├── .env.example # Environment variables template
├── .gitignore # Git exclusions
├── README.md # Main documentation
└── CHANGELOG.md # Change history
```
## File Descriptions
### Core Application Files
#### `server.js`
**Purpose**: Main Express.js application server
**Key Components**:
- Express app initialization
- Database setup and schema
- Route definitions
- WebSocket server
- Authentication middleware
- DevBench management logic
- Provision script executor
- Status monitoring (60-second interval)
**Dependencies**:
- express
- express-session
- sqlite3
- bcryptjs
- ws (WebSocket)
- child_process (for script execution)
**Key Functions**:
```javascript
- requireAuth() // Authentication middleware
- requireAdmin() // Admin authorization middleware
- executeProvisionScript() // Execute VM provisioning
- broadcastToUser() // WebSocket messaging
```
#### `config.js`
**Purpose**: Centralized configuration
**Settings**:
- Server port
- Database path
- Session configuration
- Default admin credentials
- Validation patterns
- Provision script settings
#### `provision_vm.sh`
**Purpose**: VM provisioning and management
**Commands**:
- `create <vm_name>`: Create new VM
- `status <vm_name>`: Check VM status
- `delete <vm_name>`: Delete VM
- `activate <vm_name>`: Activate VM
- `start <vm_name>`: Start VM
- `stop <vm_name>`: Stop VM
**Features**:
- SSH connection to remote host
- Output logging
- Connection info extraction
- Error handling
- Timeout management (30 minutes)
### Frontend Files
#### `views/layout.ejs`
**Purpose**: Base template for all pages
**Features**:
- Navigation bar with logo
- Help link
- Theme toggle button
- WebSocket initialization
- Bootstrap and Font Awesome imports
- Dark theme JavaScript
#### `views/dashboard.ejs`
**Purpose**: User dashboard
**Features**:
- DevBench list with cards
- Create DevBench modal
- Connection info display (SSH/VNC ports)
- Real-time log output
- Status indicators
- Copy-to-clipboard functionality
#### `views/admin.ejs`
**Purpose**: Admin panel
**Features**:
- User management table
- Add user modal
- DevBench overview table
- Password reset
- User deletion
- DevBench deletion
#### `views/help.ejs`
**Purpose**: Help and documentation
**Features**:
- SSH Config Manager guide
- Step-by-step instructions
- Download link for tool
- Connection information
- Important notes
#### `views/login.ejs`
**Purpose**: Authentication page
**Features**:
- Login form
- Error messages
- Branding
### Static Assets
#### `public/css/style.css`
**Purpose**: Custom styling
**Features**:
- NABD brand colors
- Dark theme styles
- Component styling
- Responsive design
- Smooth transitions
#### `public/images/`
**Purpose**: Image assets
**Files**:
- `nabd-logo.svg`: Light theme logo
- `nabd-logo-white.svg`: Dark theme logo
- `tbm-icon.png`: Branding icon and favicon
#### `public/downloads/`
**Purpose**: Downloadable files
**Files**:
- `db_vm_ssh_config_manager.exe`: SSH configuration tool
### Deployment Files
#### `Dockerfile`
**Purpose**: Docker image definition
**Base Image**: node:18-alpine
**Steps**:
1. Set working directory
2. Copy package files
3. Install dependencies
4. Copy application files
5. Create data/logs directories
6. Expose port 3001
7. Set start command
#### `docker-compose.yml`
**Purpose**: Container orchestration
**Configuration**:
- Service: devbench-manager
- Port mapping: 9090:3001
- Volumes: data, logs, provision script
- Networks: app-network, caddy_network
- Health check: /health endpoint
- Restart policy: unless-stopped
#### `deploy.sh`
**Purpose**: Automated deployment
**Steps**:
1. Create directories
2. Set permissions
3. Clean up old containers
4. Create Docker network
5. Build and start container
6. Wait for startup
7. Verify health
8. Display access information
### Configuration Files
#### `package.json`
**Purpose**: Node.js project configuration
**Scripts**:
- `start`: Run production server
- `dev`: Run with nodemon (auto-reload)
**Dependencies**:
- express: Web framework
- express-session: Session management
- bcryptjs: Password hashing
- sqlite3: Database
- body-parser: Request parsing
- ejs: Template engine
- ws: WebSocket
- child_process: Script execution
#### `.env.example`
**Purpose**: Environment variables template
**Variables**:
- NODE_ENV
- PORT
- SECRET_KEY
- ADMIN_EMAIL
- ADMIN_PASSWORD
#### `.gitignore`
**Purpose**: Git exclusions
**Excluded**:
- node_modules/
- data/
- logs/
- .env
- *.log
- .DS_Store
### Utility Scripts
#### `install.sh`
**Purpose**: Local installation
**Actions**:
- Check Node.js version
- Install npm dependencies
- Display start instructions
#### `start.sh`
**Purpose**: Start application
**Logic**:
- Check for Docker
- Start with Docker Compose if available
- Fall back to Node.js
- Install dependencies if needed
#### `cleanup.sh`
**Purpose**: Clean up containers
**Actions**:
- Stop containers
- Remove containers
- Prune dangling containers
- List volumes
### Documentation Files
#### `README.md`
**Purpose**: Main project documentation
**Sections**:
- Features overview
- Installation instructions
- Configuration guide
- Usage instructions
- API endpoints
- Troubleshooting
#### `CHANGELOG.md`
**Purpose**: Change history
**Content**:
- Recent updates
- SSH configuration changes
- Output format changes
- New features
- Migration notes
#### `docs/ARCHITECTURE.md`
**Purpose**: System architecture
**Content**:
- Architecture diagrams
- Component descriptions
- Data flow
- Security architecture
- Technology stack
#### `docs/DEPLOYMENT.md`
**Purpose**: Deployment guide
**Content**:
- Prerequisites
- Deployment steps
- Configuration
- Troubleshooting
- Maintenance
## Data Flow Through Files
### DevBench Creation
```
1. views/dashboard.ejs
↓ (User clicks Create)
2. server.js (POST /create-devbench)
↓ (Validates and creates DB record)
3. server.js (executeProvisionScript)
↓ (Spawns child process)
4. provision_vm.sh
↓ (SSH to remote host)
5. Remote VM Host
↓ (Creates VM, returns info)
6. provision_vm.sh
↓ (Parses output)
7. server.js
↓ (Updates database)
8. WebSocket
↓ (Broadcasts to user)
9. views/dashboard.ejs
↓ (Updates UI)
```
### Authentication
```
1. views/login.ejs
↓ (User submits form)
2. server.js (POST /login)
↓ (Validates credentials)
3. config.js (validation patterns)
↓
4. server.js (bcrypt compare)
↓ (Creates session)
5. express-session
↓ (Stores session)
6. server.js (redirect)
↓
7. views/dashboard.ejs or views/admin.ejs
```
### Theme Toggle
```
1. views/layout.ejs (Theme toggle button)
↓ (User clicks)
2. JavaScript (theme toggle handler)
↓ (Toggles class)
3. public/css/style.css (dark-theme styles)
↓ (Applies styles)
4. localStorage
↓ (Saves preference)
```
## Key Directories
### `/data`
- **Created**: On first deployment
- **Purpose**: SQLite database storage
- **Persistence**: Docker volume
- **Backup**: Copy devbench.db file
### `/logs`
- **Created**: On first deployment
- **Purpose**: Application logs
- **Persistence**: Docker volume
- **Rotation**: Manual
### `/public`
- **Purpose**: Static assets served by Express
- **Access**: Direct URL paths
- **Caching**: Browser caching enabled
### `/views`
- **Purpose**: EJS templates
- **Rendering**: Server-side
- **Access**: Via routes only
### `/docs`
- **Purpose**: Project documentation
- **Format**: Markdown
- **Access**: File system only
## Configuration Hierarchy
```
1. Environment Variables (.env)
↓
2. config.js (defaults)
↓
3. docker-compose.yml (container env)
↓
4. Application runtime
```
## Port Mapping
```
External → Container → Application
9090 → 3001 → Express Server
```
## Network Architecture
```
Internet
↓
Caddy (caddy_network)
↓
DevBench Manager (caddy_network + app-network)
↓
SQLite (local file)
↓
SSH (external)
↓
Remote VM Host
```
## File Permissions
### Scripts
- `provision_vm.sh`: 755 (executable)
- `deploy.sh`: 755 (executable)
- `install.sh`: 755 (executable)
- `start.sh`: 755 (executable)
- `cleanup.sh`: 755 (executable)
### Directories
- `data/`: 755
- `logs/`: 755
- `public/`: 755
### Database
- `devbench.db`: 644 (read/write for owner)
## Build Process
### Docker Build
```
1. Read Dockerfile
2. Pull node:18-alpine
3. Copy package.json
4. npm install
5. Copy application files
6. Set permissions
7. Create image
```
### Docker Compose
```
1. Read docker-compose.yml
2. Create networks
3. Build image (if needed)
4. Create container
5. Mount volumes
6. Start container
7. Run health checks
```
## Runtime Process
### Container Startup
```
1. Docker starts container
2. Node.js starts
3. server.js executes
4. Database initialized
5. Express server starts
6. WebSocket server starts
7. Health check passes
8. Ready for connections
```
### Request Handling
```
1. Request arrives at Caddy
2. Proxied to container
3. Express routes request
4. Middleware checks auth
5. Controller handles logic
6. Database query (if needed)
7. Template rendered
8. Response sent
```