From bb80a653462f11c602d9077d3698c199eff4a8e0 Mon Sep 17 00:00:00 2001 From: mahmamdouh Date: Sun, 28 Dec 2025 04:55:02 +0100 Subject: [PATCH] add docs --- README.md | 28 +++++++++++ doc/README.md | 22 +++++++++ doc/api_reference.md | 111 +++++++++++++++++++++++++++++++++++++++++++ doc/architecture.md | 64 +++++++++++++++++++++++++ doc/flow_diagrams.md | 75 +++++++++++++++++++++++++++++ doc/usage.md | 80 +++++++++++++++++++++++++++++++ 6 files changed, 380 insertions(+) create mode 100644 README.md create mode 100644 doc/README.md create mode 100644 doc/api_reference.md create mode 100644 doc/architecture.md create mode 100644 doc/flow_diagrams.md create mode 100644 doc/usage.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..236278d --- /dev/null +++ b/README.md @@ -0,0 +1,28 @@ +# TestArena Backend + +TestArena is an automated build and test execution system for ESP32 projects. It provides a complete workflow for cloning repositories, building firmware, and running tests in a QEMU environment, all managed through a modern web dashboard. + +## 🚀 Quick Start + +1. **Deploy**: Run `sudo ./deploy.sh` on your Ubuntu server. +2. **Access**: Open `http://:8080/` in your browser. +3. **Monitor**: Use the dashboard to track test queues and view real-time logs. + +## 📚 Documentation + +For detailed information, please refer to the documentation in the `doc/` folder: + +* **[Architecture & Design](doc/architecture.md)**: How the system is built. +* **[Usage Guide](doc/usage.md)**: How to use and manage the system. +* **[API Reference](doc/api_reference.md)**: Integration details. +* **[Workflows](doc/flow_diagrams.md)**: Visual flowcharts of key processes. + +## 🛠️ Technology Stack + +* **Backend**: FastAPI, SQLAlchemy, SQLite +* **Worker**: Python Subprocess, Bash +* **Frontend**: Vanilla HTML/JS (Glassmorphism UI) +* **Infrastructure**: Nginx, Systemd, ESP-IDF, QEMU + +--- +© 2025 TestArena Team diff --git a/doc/README.md b/doc/README.md new file mode 100644 index 0000000..ae42f3f --- /dev/null +++ b/doc/README.md @@ -0,0 +1,22 @@ +# TestArena Backend Documentation + +Welcome to the official documentation for the TestArena Backend. This system is designed to automate the build and test execution process for ESP32 projects using ESP-IDF and QEMU. + +## Table of Contents + +1. **[Architecture & Design](architecture.md)**: Overview of the system components, technology stack, and design principles. +2. **[Usage Guide](usage.md)**: Instructions on how to deploy, configure, and use the system. +3. **[API Reference](api_reference.md)**: Detailed documentation of the available REST API endpoints. +4. **[Workflows & Flowcharts](flow_diagrams.md)**: Visual representations of the system's key processes. + +## Project Overview + +TestArena is a robust platform for managing and executing automated test scenarios. It provides a web-based dashboard for monitoring test queues, real-time logging of execution steps, and detailed HTML reports of test results. + +### Key Features + +* **Automated Build & Test**: Automatically clones repositories, builds firmware, and executes tests in a QEMU environment. +* **Real-time Monitoring**: Live dashboard with search, sorting, and queue management (abort/delete). +* **Comprehensive Logging**: Timestamped and leveled logs for every step of the process. +* **Scalable Architecture**: Decoupled API and Worker services for better performance and reliability. +* **Easy Deployment**: Automated deployment script for quick setup on Ubuntu servers. diff --git a/doc/api_reference.md b/doc/api_reference.md new file mode 100644 index 0000000..bee3236 --- /dev/null +++ b/doc/api_reference.md @@ -0,0 +1,111 @@ +# API Reference + +The TestArena Backend provides a RESTful API for managing test queues and monitoring system status. + +## Base URL +`http://:8080/api` + +--- + +## Endpoints + +### 1. Submit a New Queue +`POST /queue` + +Submits a new set of tasks to the execution queue. + +**Request Body:** +```json +{ + "source": "string", + "": [ + "environment_name", + { + "": "path/to/scenario.xml" + } + ] +} +``` + +**Response:** +```json +{ + "status": "Queue OK", + "queue_id": "string" +} +``` + +--- + +### 2. List All Queues +`GET /queues` + +Returns a list of all queues in the system, ordered by creation date (newest first). + +**Response:** +```json +[ + { + "id": "string", + "status": "Waiting|Running|Finished|Aborted", + "created_at": "ISO8601 Timestamp", + "environment": "string", + "source": "string" + } +] +``` + +--- + +### 3. Get Status +`GET /status/{id}` + +Gets the status of a specific queue or task. + +**Response:** +```json +{ + "id": "string", + "type": "queue|task", + "status": "string" +} +``` + +--- + +### 4. Abort Queue or Task +`POST /abort/{id}` + +Aborts a waiting or running queue or a single task. + +**Response:** +```json +{ + "id": "string", + "status": "Aborted" +} +``` + +--- + +### 5. Delete Queue +`DELETE /delete/{id}` + +Permanently deletes a queue, its associated tasks, and all related files from the server. + +**Response:** +```json +{ + "id": "string", + "status": "Deleted" +} +``` + +--- + +## Error Handling + +The API uses standard HTTP status codes: +* `200 OK`: Request successful. +* `404 Not Found`: The requested ID does not exist. +* `500 Internal Server Error`: An unexpected error occurred on the server. diff --git a/doc/architecture.md b/doc/architecture.md new file mode 100644 index 0000000..1138713 --- /dev/null +++ b/doc/architecture.md @@ -0,0 +1,64 @@ +# Architecture & Design + +This document describes the high-level architecture and design of the TestArena Backend. + +## System Architecture + +TestArena follows a decoupled architecture consisting of a RESTful API and a background worker. + +```mermaid +graph TD + User((User)) -->|HTTP| API[FastAPI Web Server] + API -->|Read/Write| DB[(SQLite Database)] + API -->|Create| FS[Filesystem /testarena] + Worker[Background Worker] -->|Poll| DB + Worker -->|Execute| Scripts[Execution Scripts] + Scripts -->|Build/Test| QEMU[QEMU Emulator] + Scripts -->|Write Logs| FS + User -->|View| Dashboard[Web Dashboard] + Dashboard -->|API Calls| API +``` + +## Core Components + +### 1. FastAPI Web Server (`main.py`) +The entry point for the system. It handles: +* Receiving test queue requests via REST API. +* Managing the SQLite database. +* Serving the web dashboard and static files. +* Providing endpoints for status monitoring, aborting, and deleting queues. + +### 2. Background Worker (`worker.py`) +A dedicated process that continuously polls the database for "Waiting" queues. Its responsibilities include: +* Cloning and checking out the correct branch of the target repository. +* Orchestrating the build process using `idf.py build`. +* Running the QEMU simulation. +* Executing individual test scenarios. +* Managing real-time logging to the filesystem. + +### 3. Execution Scripts (`TPF/`) +A set of specialized scripts for different stages of the workflow: +* `gitea_repo_controller.sh`: Handles Git operations (clone, checkout, pull). +* `scenario_execution.py`: Parses XML scenarios and runs test suites. +* `test_execution.sh`: Executes individual test commands and generates HTML logs. +* `scenario_exe_parser.py`: Helper script for parsing XML scenario files. + +## Technology Stack + +| Layer | Technology | +| :--- | :--- | +| **Backend Framework** | FastAPI (Python 3.12+) | +| **Database** | SQLite with SQLAlchemy ORM | +| **Frontend** | Vanilla HTML5, CSS3 (Modern Glassmorphism UI), JavaScript | +| **Process Management** | Systemd (testarena-app, testarena-worker) | +| **Web Server / Proxy** | Nginx | +| **Build System** | ESP-IDF (Espressif IoT Development Framework) | +| **Emulator** | QEMU (XTENSA) | +| **Scripting** | Bash, Python | + +## Design Principles + +* **Decoupling**: The API and Worker are separate processes, allowing the UI to remain responsive even during heavy test execution. +* **Real-time Visibility**: All subprocess output is streamed in real-time to both the console and log files. +* **Robustness**: Implemented global timeouts, automatic cleanup of orphaned processes, and detailed error handling. +* **Automation**: The entire deployment and execution flow is fully automated to minimize manual intervention. diff --git a/doc/flow_diagrams.md b/doc/flow_diagrams.md new file mode 100644 index 0000000..dd711fb --- /dev/null +++ b/doc/flow_diagrams.md @@ -0,0 +1,75 @@ +# Workflows & Flowcharts + +This document provides visual representations of the key processes within the TestArena system. + +## 1. Test Queue Submission Flow + +This flowchart shows the process from when a user submits a queue until it is ready for the worker. + +```mermaid +sequenceDiagram + participant User + participant API as FastAPI (main.py) + participant DB as SQLite + participant FS as Filesystem + + User->>API: POST /api/queue (Payload) + API->>API: Extract Queue ID, Source, Tasks + API->>FS: Create /testarena/ folder + API->>FS: Create queue_status.json + API->>DB: Insert Queue & Task records + API-->>User: 200 OK (Queue OK) +``` + +## 2. Worker Execution Workflow + +This diagram illustrates the lifecycle of a test queue as processed by the background worker. + +```mermaid +flowchart TD + Start([Start Worker]) --> Poll{Poll DB for 'Waiting'} + Poll -- No --> Wait[Wait 5s] --> Poll + Poll -- Yes --> Running[Set Status to 'Running'] + Running --> Clone[Clone/Checkout Repo] + Clone --> Build[Build Firmware: idf.py build] + Build --> QEMU[Start QEMU: idf.py qemu] + QEMU --> Loop[Loop through Tasks] + + subgraph Task Execution + Loop --> RunScript[Run scenario_execution.py] + RunScript --> Stream[Stream Output to Log] + Stream --> Parse[Parse Results] + Parse --> UpdateDB[Update Task Status & Result] + end + + UpdateDB --> Next{More Tasks?} + Next -- Yes --> Loop + Next -- No --> Finish[Set Queue to 'Finished'] + Finish --> Wait +``` + +## 3. Real-time Logging Architecture + +How logs are captured and displayed to the user. + +```mermaid +graph LR + Sub[Subprocess] -->|Stdout/Stderr| Pipe[Pipe] + Pipe -->|Read Line| Worker[worker.py] + Worker -->|Write| File[queue_log.txt] + Worker -->|Print| Console[Systemd Journal] + User -->|View| Dashboard[Web Dashboard] + Dashboard -->|Fetch| API[FastAPI] + API -->|Read| File +``` + +## 4. Delete Queue Workflow + +The process of cleaning up system data. + +```mermaid +flowchart LR + Req[DELETE /api/delete/{id}] --> DB[Delete DB Records] + DB --> FS[Remove /testarena/{id} Directory] + FS --> Res[Return Success] +``` diff --git a/doc/usage.md b/doc/usage.md new file mode 100644 index 0000000..b13beba --- /dev/null +++ b/doc/usage.md @@ -0,0 +1,80 @@ +# Usage Guide + +This guide provides instructions on how to deploy, configure, and use the TestArena system. + +## Deployment + +The system is designed to be deployed on an Ubuntu server. An automated deployment script is provided. + +### Prerequisites +* Ubuntu 22.04 or later. +* ESP-IDF installed at `/home/asf/esp/esp-idf`. +* QEMU (XTENSA) installed and available in the system path. + +### Installation Steps +1. Clone the repository to `/home/asf/testarena_backend`. +2. Navigate to the directory: `cd /home/asf/testarena_backend`. +3. Run the deployment script with sudo: + ```bash + sudo ./deploy.sh + ``` + This script will: + * Install system dependencies (`nginx`, `sqlite3`, `perl`, etc.). + * Set up a Python virtual environment and install requirements. + * Configure Nginx as a reverse proxy. + * Set up and enable systemd services for the API and Worker. + +## Managing Services + +Use standard `systemctl` commands to manage the TestArena services: + +```bash +# Restart services +sudo systemctl restart testarena-app testarena-worker + +# Check status +sudo systemctl status testarena-app testarena-worker + +# View logs +sudo journalctl -u testarena-worker -f +``` + +## Using the Dashboard + +Access the dashboard at `http://:8080/`. + +### Features: +* **Queue Monitor**: View all test queues, their status, and environment. +* **Search**: Filter queues by Queue ID using the search box. +* **Sorting**: Click on table headers (Queue ID, Environment, Status) to sort the data. +* **Actions**: + * **Abort**: Stop a running or waiting queue. + * **Delete**: Permanently remove a queue's data from the database and filesystem. +* **Live Logs**: View real-time system logs in the sidebar. + +## Submitting a Test Queue + +You can submit a new test queue by sending a POST request to `/api/queue`. + +### Example Payload: +```json +{ + "source": "feature/new-sensor", + "QUEUE_12345": [ + "Production_Env", + { + "TASK_001": "path/to/scenario_1.xml", + "TASK_002": "path/to/scenario_2.xml" + } + ] +} +``` + +## Viewing Results + +Test results are stored in `/home/asf/testarena//`. +* `queue_log.txt`: The full execution log for the entire queue. +* `/execution_report.html`: A detailed HTML report for a specific task. +* `/-logging.html`: Individual logs for each test case. + +You can also browse results via the web interface at `http://:8080/results/`.