ASF_01_sys_sw_arch/1 software design/components/logger/COMPONENT.md

Logger Component Specification

Component ID: C-LOGGER-001
Component Name: Logger
Version: 1.0
Date: 2025-02-01
Location: application_layer/utils/logger/

1. Component Overview

1.1 Purpose

The Logger component provides centralized logging infrastructure for the ASF Sensor Hub, enabling comprehensive system diagnostics, debugging, and operational monitoring. It supports multiple log levels, output destinations, and efficient log management with minimal performance impact.

1.2 Scope

• Multi-level logging system (DEBUG, INFO, WARN, ERROR, FATAL)
• Multiple output destinations (UART, file, network, memory buffer)
• Log formatting and timestamping
• Log rotation and storage management
• Runtime log level configuration
• Thread-safe logging operations

1.3 Responsibilities

• Provide unified logging interface for all system components
• Format and timestamp log messages
• Route log messages to appropriate output destinations
• Manage log file rotation and storage limits
• Support runtime log level configuration
• Ensure thread-safe logging operations

2. Component Architecture

2.1 Static View

graph TB
    subgraph "Logger"
        LC[Log Controller]
        LF[Log Formatter]
        LR[Log Router]
        LM[Log Manager]
        LB[Log Buffer]
    end
    
    subgraph "Output Destinations"
        UART[UART Output]
        FILE[File Output]
        NET[Network Output]
        MEM[Memory Buffer]
    end
    
    subgraph "External Components"
        COMPS[System Components]
        FS[File System]
        CM[Communication Manager]
        TU[Time Utils]
    end
    
    LC --> LF
    LC --> LR
    LC --> LM
    LC --> LB
    
    LR --> UART
    LR --> FILE
    LR --> NET
    LR --> MEM
    
    COMPS --> LC
    FILE --> FS
    NET --> CM
    LF --> TU

2.2 Internal Components

2.2.1 Log Controller

• Central coordination of logging operations
• Log level filtering and management
• Component lifecycle and configuration

2.2.2 Log Formatter

• Log message formatting and structuring
• Timestamp generation and formatting
• Message metadata enrichment

2.2.3 Log Router

• Route log messages to appropriate destinations
• Output destination management
• Load balancing and failover

2.2.4 Log Manager

• Log file rotation and cleanup
• Storage space management
• Log retention policy enforcement

2.2.5 Log Buffer

• In-memory log buffering for performance
• Asynchronous log processing
• Buffer overflow handling

3. Interfaces

3.1 Provided Interfaces

3.1.1 ILogger

class ILogger {
public:
    virtual ~ILogger() = default;
    
    // Logging Methods
    virtual void debug(const std::string& message, const std::string& component = "") = 0;
    virtual void info(const std::string& message, const std::string& component = "") = 0;
    virtual void warn(const std::string& message, const std::string& component = "") = 0;
    virtual void error(const std::string& message, const std::string& component = "") = 0;
    virtual void fatal(const std::string& message, const std::string& component = "") = 0;
    
    // Formatted Logging
    virtual void logf(LogLevel level, const std::string& component, 
                     const char* format, ...) = 0;
    virtual void log(LogLevel level, const std::string& component, 
                    const std::string& message) = 0;
    
    // Structured Logging
    virtual void logStructured(LogLevel level, const std::string& component,
                              const std::string& message,
                              const std::map<std::string, std::string>& metadata) = 0;
    
    // Configuration
    virtual Result<void> setLogLevel(LogLevel level) = 0;
    virtual LogLevel getLogLevel() const = 0;
    virtual Result<void> setComponentLogLevel(const std::string& component, LogLevel level) = 0;
    virtual LogLevel getComponentLogLevel(const std::string& component) const = 0;
    
    // Output Management
    virtual Result<void> addOutput(std::unique_ptr<ILogOutput> output) = 0;
    virtual Result<void> removeOutput(const std::string& output_name) = 0;
    virtual std::vector<std::string> getActiveOutputs() const = 0;
    
    // System Management
    virtual Result<void> initialize() = 0;
    virtual Result<void> shutdown() = 0;
    virtual Result<void> flush() = 0;
    virtual LogStatistics getStatistics() const = 0;
};

3.1.2 ILogOutput

class ILogOutput {
public:
    virtual ~ILogOutput() = default;
    virtual Result<void> writeLog(const LogEntry& entry) = 0;
    virtual Result<void> flush() = 0;
    virtual std::string getName() const = 0;
    virtual bool isAvailable() const = 0;
    virtual LogOutputStatistics getStatistics() const = 0;
};

3.1.3 ILogFormatter

class ILogFormatter {
public:
    virtual ~ILogFormatter() = default;
    virtual std::string formatLogEntry(const LogEntry& entry) const = 0;
    virtual Result<void> setFormat(const std::string& format_string) = 0;
    virtual std::string getFormat() const = 0;
};

3.2 Required Interfaces

3.2.1 ITimeUtils

• Timestamp generation for log entries
• Time formatting for log messages
• Duration tracking for performance logging

3.2.2 IFileSystem

• Log file creation and management
• File rotation and cleanup operations
• Storage space monitoring

3.2.3 ICommunicationManager

• Network log transmission
• Remote logging capabilities
• Log aggregation support

3.2.4 IConfigurationManager

• Log level configuration
• Output destination configuration
• Log format settings

4. Dynamic View

4.1 Log Message Processing Sequence

sequenceDiagram
    participant COMP as System Component
    participant LC as Log Controller
    participant LF as Log Formatter
    participant LR as Log Router
    participant OUT as Log Output
    participant TU as Time Utils
    
    COMP->>LC: info("Sensor data acquired", "SensorManager")
    LC->>LC: checkLogLevel(INFO, "SensorManager")
    
    alt Log Level Enabled
        LC->>TU: getCurrentTimestamp()
        TU-->>LC: timestamp
        LC->>LF: formatLogEntry(entry)
        LF->>LF: applyFormat(entry, timestamp)
        LF-->>LC: formatted_message
        LC->>LR: routeLog(formatted_message)
        
        par Parallel Output
            LR->>OUT: writeLog(formatted_message)
            OUT-->>LR: write_result
        end
        
        LC->>LC: updateStatistics()
    else Log Level Disabled
        LC->>LC: discardMessage()
    end

4.2 Log Rotation Sequence

sequenceDiagram
    participant LM as Log Manager
    participant FS as File System
    participant LC as Log Controller
    
    LM->>FS: checkLogFileSize()
    FS-->>LM: current_size
    
    alt Size Limit Exceeded
        LM->>LC: pauseLogging()
        LM->>FS: rotateLogFile()
        FS->>FS: renameCurrentLog()
        FS->>FS: createNewLogFile()
        FS->>FS: deleteOldLogs()
        FS-->>LM: rotation_complete
        LM->>LC: resumeLogging()
    end

4.3 Asynchronous Logging Sequence

sequenceDiagram
    participant COMP as Component
    participant LB as Log Buffer
    participant LC as Log Controller
    participant OUT as Output
    
    COMP->>LB: log(message)
    LB->>LB: addToBuffer(message)
    LB-->>COMP: buffered (non-blocking)
    
    Note over LB,OUT: Background Processing
    
    loop Buffer Processing
        LB->>LC: getNextLogEntry()
        LC->>OUT: writeLog(entry)
        OUT-->>LC: write_complete
        LB->>LB: removeFromBuffer()
    end

5. Log Levels and Filtering

5.1 Log Levels

enum class LogLevel {
    DEBUG = 0,      // Detailed debugging information
    INFO = 1,       // General information messages
    WARN = 2,       // Warning conditions
    ERROR = 3,      // Error conditions
    FATAL = 4,      // Fatal error conditions
    OFF = 5         // Logging disabled
};

5.2 Log Level Management

class LogLevelManager {
public:
    void setGlobalLogLevel(LogLevel level);
    LogLevel getGlobalLogLevel() const;
    void setComponentLogLevel(const std::string& component, LogLevel level);
    LogLevel getComponentLogLevel(const std::string& component) const;
    bool isLogLevelEnabled(LogLevel level, const std::string& component) const;
    
private:
    LogLevel global_level_;
    std::map<std::string, LogLevel> component_levels_;
};

5.3 Runtime Configuration

• Global Log Level: System-wide minimum log level
• Component-Specific Levels: Per-component log level overrides
• Dynamic Adjustment: Runtime log level changes without restart
• Configuration Persistence: Log level settings saved to configuration

6. Log Formatting

6.1 Log Entry Structure

struct LogEntry {
    LogLevel level;
    std::string component;
    std::string message;
    uint64_t timestamp_ms;
    uint32_t thread_id;
    std::string file_name;
    uint32_t line_number;
    std::map<std::string, std::string> metadata;
    uint64_t sequence_number;
};

6.2 Format Templates

namespace LogFormats {
    // Standard format: [TIMESTAMP] [LEVEL] [COMPONENT] MESSAGE
    constexpr const char* STANDARD = "[%t] [%l] [%c] %m";
    
    // Detailed format: [TIMESTAMP] [LEVEL] [COMPONENT] [FILE:LINE] MESSAGE
    constexpr const char* DETAILED = "[%t] [%l] [%c] [%f:%n] %m";
    
    // Compact format: %t %l %c %m
    constexpr const char* COMPACT = "%t %l %c %m";
    
    // JSON format for structured logging
    constexpr const char* JSON = "{\"timestamp\":\"%t\",\"level\":\"%l\",\"component\":\"%c\",\"message\":\"%m\"}";
}

6.3 Format Specifiers

• %t - Timestamp
• %l - Log level
• %c - Component name
• %m - Message
• %f - File name
• %n - Line number
• %i - Thread ID
• %s - Sequence number

7. Output Destinations

7.1 UART Output

class UARTLogOutput : public ILogOutput {
public:
    UARTLogOutput(UARTPort port, uint32_t baud_rate);
    Result<void> writeLog(const LogEntry& entry) override;
    Result<void> flush() override;
    bool isAvailable() const override;
    
private:
    UARTPort uart_port_;
    uint32_t baud_rate_;
    bool is_initialized_;
};

7.2 File Output

class FileLogOutput : public ILogOutput {
public:
    FileLogOutput(const std::string& file_path, size_t max_file_size);
    Result<void> writeLog(const LogEntry& entry) override;
    Result<void> flush() override;
    Result<void> rotateFile();
    
private:
    std::string file_path_;
    size_t max_file_size_;
    size_t current_file_size_;
    std::unique_ptr<std::ofstream> file_stream_;
};

7.3 Network Output

class NetworkLogOutput : public ILogOutput {
public:
    NetworkLogOutput(const std::string& server_url, uint16_t port);
    Result<void> writeLog(const LogEntry& entry) override;
    Result<void> flush() override;
    
private:
    std::string server_url_;
    uint16_t port_;
    std::queue<LogEntry> pending_logs_;
};

7.4 Memory Buffer Output

class MemoryLogOutput : public ILogOutput {
public:
    MemoryLogOutput(size_t buffer_size);
    Result<void> writeLog(const LogEntry& entry) override;
    std::vector<LogEntry> getRecentLogs(size_t count) const;
    void clearBuffer();
    
private:
    std::deque<LogEntry> log_buffer_;
    size_t max_buffer_size_;
    mutable std::mutex buffer_mutex_;
};

8. Log Management

8.1 Log Rotation

struct LogRotationConfig {
    size_t max_file_size;           // Maximum size per log file
    uint32_t max_file_count;        // Maximum number of log files
    std::chrono::hours max_age;     // Maximum age of log files
    bool compress_old_files;        // Compress rotated files
    std::string rotation_pattern;   // File naming pattern
};

8.2 Storage Management

• Size-Based Rotation: Rotate logs when file size exceeds limit
• Time-Based Rotation: Rotate logs based on time intervals
• Count-Based Cleanup: Maintain maximum number of log files
• Compression: Compress old log files to save space
• Cleanup: Automatic deletion of old log files

8.3 Log Retention Policy

• Retention Period: Configurable log retention duration
• Priority-Based Retention: Keep ERROR/FATAL logs longer
• Storage Limits: Respect available storage space
• Emergency Cleanup: Free space when storage is full

9. Performance Optimization

9.1 Asynchronous Logging

class AsyncLogger {
public:
    void logAsync(const LogEntry& entry);
    void flush();
    void setBufferSize(size_t size);
    
private:
    std::queue<LogEntry> log_queue_;
    std::thread worker_thread_;
    std::condition_variable cv_;
    std::mutex queue_mutex_;
    bool shutdown_requested_;
};

9.2 Performance Features

• Lock-Free Queues: Minimize contention in multi-threaded environment
• Batch Processing: Process multiple log entries together
• Memory Pools: Pre-allocated memory for log entries
• Lazy Formatting: Format messages only when needed
• Compile-Time Filtering: Remove debug logs in release builds

9.3 Resource Management

• Memory Limits: Bounded memory usage for log buffers
• CPU Throttling: Limit CPU usage for log processing
• I/O Optimization: Batch file writes for efficiency
• Network Buffering: Buffer network logs to reduce overhead

10. Configuration

10.1 Logger Configuration

struct LoggerConfig {
    // Global Settings
    LogLevel global_log_level;
    std::map<std::string, LogLevel> component_log_levels;
    std::string log_format;
    
    // Output Configuration
    std::vector<LogOutputConfig> outputs;
    bool async_logging_enabled;
    size_t async_buffer_size;
    
    // File Logging Configuration
    std::string log_file_path;
    LogRotationConfig rotation_config;
    
    // Network Logging Configuration
    std::string log_server_url;
    uint16_t log_server_port;
    bool network_logging_enabled;
    
    // Performance Configuration
    size_t max_message_length;
    std::chrono::milliseconds flush_interval;
    bool compile_time_filtering;
};

11. Thread Safety

11.1 Synchronization Strategy

• Lock-Free Queues: For high-performance logging
• Mutex Protection: For shared data structures
• Thread-Local Storage: For per-thread log contexts
• Atomic Operations: For counters and flags

11.2 Concurrency Considerations

• Multiple Writers: Support concurrent logging from multiple threads
• Reader-Writer Separation: Separate read and write operations
• Deadlock Prevention: Careful lock ordering and timeout handling
• Performance Impact: Minimize synchronization overhead

12. Error Handling

12.1 Logging Errors

enum class LogError {
    OUTPUT_UNAVAILABLE,     // Log output destination unavailable
    BUFFER_FULL,           // Log buffer overflow
    FORMAT_ERROR,          // Log format string error
    FILE_WRITE_ERROR,      // File write operation failed
    NETWORK_ERROR,         // Network transmission failed
    CONFIGURATION_ERROR    // Invalid configuration
};

12.2 Error Recovery

• Fallback Outputs: Switch to alternative outputs on failure
• Buffer Overflow: Drop oldest messages when buffer is full
• File Errors: Create new files or switch to memory logging
• Network Errors: Queue messages for later transmission

13. Performance Characteristics

13.1 Timing Requirements

• Log Entry Processing: < 1ms for standard log entry
• Asynchronous Logging: < 100μs for queue insertion
• File Write Operations: Batched for efficiency
• Network Transmission: Asynchronous with buffering

13.2 Resource Usage

• Memory: Configurable buffer sizes (default 64KB)
• Storage: Configurable with rotation and cleanup
• CPU: < 2% average utilization for logging operations
• Network: Minimal bandwidth for remote logging

14. Security Considerations

14.1 Log Security

• Sensitive Data Filtering: Prevent logging of sensitive information
• Access Control: Restrict access to log files and outputs
• Log Integrity: Protect logs from tampering
• Secure Transmission: Encrypt network log transmission

14.2 Privacy Protection

• Data Sanitization: Remove or mask sensitive data
• Retention Limits: Automatic deletion of old logs
• Access Logging: Log access to log files
• Audit Trail: Maintain audit trail of log operations

15. Testing Strategy

15.1 Unit Tests

• Log level filtering correctness
• Message formatting accuracy
• Output destination functionality
• Configuration validation

15.2 Integration Tests

• Multi-threaded logging scenarios
• Log rotation and cleanup
• Network logging functionality
• Performance under load

15.3 Performance Tests

• Logging throughput measurement
• Memory usage validation
• CPU impact assessment
• Storage efficiency testing

16. Dependencies

16.1 Internal Dependencies

• Time Utils for timestamp generation
• File System for log file management
• Communication Manager for network logging
• Configuration Manager for settings

16.2 External Dependencies

• ESP-IDF UART drivers
• File system support (SPIFFS/FAT)
• Network stack for remote logging
• Threading and synchronization primitives

17. Constraints and Assumptions

17.1 Constraints

• Memory usage must be bounded and configurable
• Logging must not significantly impact system performance
• Log files must respect storage space limitations
• Thread safety required for multi-threaded environment

17.2 Assumptions

• Sufficient storage space for log files
• Network connectivity for remote logging (when enabled)
• Proper file system configuration
• Adequate system resources for logging operations

---

Document Status: Final for Implementation
Dependencies: Time Utils, File System, Communication Manager, Configuration Manager
Next Review: After component implementation and performance validation