testarena_backend/doc/architecture.md

Architecture & Design

This document describes the high-level architecture and design of the TestArena Backend.

System Architecture

TestArena follows a decoupled architecture consisting of a RESTful API and a background worker.

graph TD
    User((User)) -->|HTTP| API[FastAPI Web Server]
    API -->|Read/Write| DB[(SQLite Database)]
    API -->|Create| FS[Filesystem /testarena]
    Worker[Background Worker] -->|Poll| DB
    Worker -->|Execute| Scripts[Execution Scripts]
    Scripts -->|Build/Test| QEMU[QEMU Emulator]
    Scripts -->|Write Logs| FS
    User -->|View| Dashboard[Web Dashboard]
    Dashboard -->|API Calls| API

Core Components

1. FastAPI Web Server (main.py)

The entry point for the system. It handles:

• Receiving test queue requests via REST API.
• Managing the SQLite database.
• Serving the web dashboard and static files.
• Providing endpoints for status monitoring, aborting, and deleting queues.

2. Background Worker (worker.py)

A dedicated process that continuously polls the database for "Waiting" queues. Its responsibilities include:

• Cloning and checking out the correct branch of the target repository.
• Orchestrating the build process using idf.py build.
• Running the QEMU simulation.
• Executing individual test scenarios.
• Managing real-time logging to the filesystem.

3. Execution Scripts (TPF/)

A set of specialized scripts for different stages of the workflow:

• gitea_repo_controller.sh: Handles Git operations (clone, checkout, pull).
• scenario_execution.py: Parses XML scenarios and runs test suites.
• test_execution.sh: Executes individual test commands and generates HTML logs.
• scenario_exe_parser.py: Helper script for parsing XML scenario files.

Technology Stack

Layer	Technology
Backend Framework	FastAPI (Python 3.12+)
Database	SQLite with SQLAlchemy ORM
Frontend	Vanilla HTML5, CSS3 (Modern Glassmorphism UI), JavaScript
Process Management	Systemd (testarena-app, testarena-worker)
Web Server / Proxy	Nginx
Build System	ESP-IDF (Espressif IoT Development Framework)
Emulator	QEMU (XTENSA)
Scripting	Bash, Python

Design Principles

• Decoupling: The API and Worker are separate processes, allowing the UI to remain responsive even during heavy test execution.
• Real-time Visibility: All subprocess output is streamed in real-time to both the console and log files.
• Robustness: Implemented global timeouts, automatic cleanup of orphaned processes, and detailed error handling.
• Automation: The entire deployment and execution flow is fully automated to minimize manual intervention.