testarena/doc/architecture.md

TestArena Architecture

TestArena is a web-based platform for managing and executing automated tests on remote hardware setups. It acts as a central hub for users to submit test jobs, monitor execution, and view results.

System Overview

The system consists of three main layers:

1. Web Frontend (Flask):

   • Provides the user interface for job submission, dashboard monitoring, and administration.
   • Handles user authentication and input validation.
   • Exposes REST APIs for programmatic access.

2. Backend Logic (Python/Flask):

   • Job Management: Manages the lifecycle of test jobs (Waiting -> In Progress -> Passed/Failed).
   • Remote Execution: Communicates with the remote test server via HTTP APIs to trigger queues and poll status.
   • Database: Uses SQLite (via SQLAlchemy) to store user data, job history, and results.

3. Remote Test Server (External):

   • Executes the actual test scenarios on hardware/simulators.
   • Exposes APIs for queue management (/api/queue, /api/status, /api/abort).
   • Hosts the test results and logs.

Key Components

1. Job Processing Flow

1. Submission: User submits a job via UI or API (POST /api/submit_job).
2. Validation: System validates credentials and branch existence (via SSH).
3. Queueing: A local Job record is created. The system sends a payload to the Remote Server's /api/queue endpoint.
4. Monitoring: A background thread in the Flask app polls the Remote Server every 20 seconds.
   • Checks Queue Status (/api/status/<queue_id>).
   • Checks Task Status (/api/status/<task_id>).
   • Updates local DB with progress and results.
5. Completion:
   • When all tasks finish, the job is marked as passed or failed.
   • Cleanup: An SSH command is automatically executed to clean up the remote workspace.
   • Timeout: If a job runs > 1 hour, it is auto-aborted.

2. Database Schema

• Users: id, username, password_hash, is_admin
• Jobs: id, user_id, branch_name, scenarios, status, remote_queue_id, remote_results, queue_log

3. API Layer

• POST /api/submit_job: Programmatic job submission.
• GET /api/job/<id>: Status retrieval.

Deployment

• Containerization: Docker & Docker Compose.
• Web Server: Gunicorn (WSGI) behind Caddy (Reverse Proxy/SSL).
• Database: SQLite (Persistent volume).

Security

• Authentication: Flask-Login for UI, Basic Auth for API.
• SSH: Uses sshpass with strict host checking disabled (internal network) for remote operations.
• Environment: Secrets managed via .env file.