refactor: [#14] separate testing concerns across three-layer architecture

- Refactor infrastructure/tests/test-ci.sh to only test infrastructure concerns
- Remove global 'make lint' calls from infrastructure layer
- Create new application/tests/test-ci.sh for application-only testing
- Create new tests/test-ci.sh for project-wide orchestration
- Update Makefile with proper layer separation (infra-test-ci, app-test-ci, test-ci)
- Update GitHub workflow to use 'make test-ci' for complete validation
- Fix application tests to be CI-friendly (optional environment files)
- Add comprehensive documentation in .github/copilot-instructions.md
- Document three-layer testing architecture with clear boundaries
- Add AI assistant guidelines to prevent future violations

Testing Hierarchy:
- make test-ci: Project-wide orchestrator (global + infra + app)
- make infra-test-ci: Infrastructure layer only
- make app-test-ci: Application layer only

Prevents mixing concerns like calling global commands from layer-specific tests.

Author: josecelano

.github/copilot-instructions.md

@@ -348,6 +348,66 @@ The project includes a comprehensive linting script that validates all file type
 - **No secrets**: Never commit SSH keys, passwords, or tokens
 - **Documentation**: Update docs for any infrastructure changes
 
+#### Three-Layer Testing Architecture
+
+**CRITICAL**: This project uses a strict three-layer testing architecture.
+**Never mix concerns across layers**:
+
+**1. Project-Wide/Global Layer** (`tests/` folder)
+
+- **Command**: `make test-ci`
+- **Scope**: Cross-cutting concerns that span all layers
+- **Responsibilities**:
+  - Global syntax validation (`make lint` / `./scripts/lint.sh`)
+  - Project structure validation (`tests/test-unit-project.sh`)
+  - Makefile validation
+  - Orchestrates infrastructure and application layer tests
+
+**2. Infrastructure Layer** (`infrastructure/` folder)
+
+- **Command**: `make infra-test-ci`
+- **Scope**: Infrastructure-specific validation ONLY
+- **Responsibilities**:
+  - Terraform/OpenTofu syntax validation
+  - Cloud-init template validation
+  - Infrastructure script validation
+  - Infrastructure configuration validation
+- **NEVER**: Call global commands like `make lint` from infrastructure tests
+
+**3. Application Layer** (`application/` folder)
+
+- **Command**: `make app-test-ci`
+- **Scope**: Application-specific validation ONLY
+- **Responsibilities**:
+  - Docker Compose syntax validation
+  - Application configuration validation
+  - Deployment script validation
+  - Grafana dashboard validation
+- **NEVER**: Call global commands like `make lint` from application tests
+
+**Testing Hierarchy:**
+
+```text
+make test-ci (Project-wide orchestrator)
+├── Global concerns (syntax, structure, Makefile)
+├── make infra-test-ci (Infrastructure layer only)
+└── make app-test-ci (Application layer only)
+```
+
+**Common Mistake to Avoid:**
+
+- ❌ Adding `make lint` calls inside `infrastructure/tests/test-ci.sh`
+- ❌ Testing application concerns from infrastructure tests
+- ❌ Testing infrastructure concerns from application tests
+- ✅ Keep each layer focused on its own responsibilities only
+- ✅ Use `make test-ci` for complete validation that orchestrates all layers
+
+**GitHub Actions Integration:**
+
+- Uses `make test-ci` (not `make infra-test-ci`) to ensure all layers are tested
+- Runs without virtualization (CI-compatible)
+- Maintains separation of concerns for maintainable testing
+
 #### End-to-End Smoke Testing
 
 For verifying the functionality of the tracker from an end-user's perspective (e.g., simulating announce/scrape requests), refer to the **Smoke Testing Guide**. This guide explains how to use the official `torrust-tracker-client` tools to perform black-box testing against a running tracker instance without needing a full BitTorrent client.
@@ -466,6 +526,27 @@ When providing assistance:
 - Test infrastructure changes locally before suggesting them
 - Provide clear explanations and documentation
 - Consider the migration to Hetzner infrastructure in suggestions
+- **CRITICAL**: Respect the three-layer testing architecture (see Testing Requirements above)
+
+#### Testing Layer Separation (CRITICAL)
+
+**NEVER** mix testing concerns across the three layers:
+
+- **Infrastructure tests** (`infrastructure/tests/`) should ONLY test infrastructure concerns
+- **Application tests** (`application/tests/`) should ONLY test application concerns
+- **Global tests** (`tests/`) handle cross-cutting concerns and orchestration
+
+**Common violations to avoid:**
+
+- ❌ Adding `make lint` calls in `infrastructure/tests/test-ci.sh`
+- ❌ Testing Docker Compose from infrastructure layer
+- ❌ Testing Terraform from application layer
+- ❌ Any layer calling commands from other layers directly
+
+**Always use the proper orchestrator:**
+
+- Use `make test-ci` for complete testing (orchestrates all layers)
+- Use layer-specific commands only when targeting that specific layer
 
 #### Command Execution Context
 

.github/workflows/testing.yml

@@ -27,4 +27,4 @@ jobs:
           sudo ./install-opentofu.sh --install-method deb
 
       - name: Run CI test suite
-        run: make infra-test-ci
+        run: make test-ci

Makefile

@@ -35,8 +35,10 @@ help: ## Show this help message
 	@echo "🖥️  VM ACCESS:"
 	@awk 'BEGIN {FS = ":.*?## "} /^vm-.*:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
 	@echo ""
-	@echo "🧪 GLOBAL TESTING:"
+	@echo "🧪 TESTING (3-LAYER ARCHITECTURE):"
 	@awk 'BEGIN {FS = ":.*?## "} /^test.*:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
+	@awk 'BEGIN {FS = ":.*?## "} /^infra-test.*:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
+	@awk 'BEGIN {FS = ":.*?## "} /^app-test.*:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
 	@echo ""
 	@echo "⚙️  SYSTEM SETUP:"
 	@awk 'BEGIN {FS = ":.*?## "} /^(install-deps|clean).*:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@@ -99,9 +101,10 @@ infra-test-prereq: ## Test system prerequisites for development
 	@echo "Testing prerequisites..."
 	$(INFRA_TESTS_DIR)/test-unit-infrastructure.sh vm-prereq
 
-infra-test-ci: ## Run infrastructure CI-compatible tests
-	@echo "Running infrastructure CI-compatible tests..."
-	$(INFRA_TESTS_DIR)/test-ci.sh
+infra-test-ci: ## Run infrastructure-only CI tests (no global concerns)
+	@echo "Running infrastructure-only CI tests..."
+	$(INFRA_TESTS_DIR)/test-unit-config.sh
+	$(INFRA_TESTS_DIR)/test-unit-scripts.sh
 
 infra-test-local: ## Run local-only infrastructure tests (requires virtualization)
 	@echo "Running local-only infrastructure tests..."
@@ -135,6 +138,10 @@ app-test-services: ## Test application services
 	@echo "Testing application services..."
 	$(TESTS_DIR)/test-unit-application.sh services
 
+app-test-ci: ## Run application-only CI tests (no global concerns)
+	@echo "Running application-only CI tests..."
+	application/tests/test-ci.sh
+
 # =============================================================================
 # VM ACCESS AND DEBUGGING
 # =============================================================================
@@ -212,6 +219,16 @@ test-e2e: ## Run comprehensive end-to-end test (follows integration guide)
 	@echo "Running comprehensive end-to-end test..."
 	$(TESTS_DIR)/test-e2e.sh $(ENVIRONMENT)
 
+test-ci: ## Run project-wide CI tests (global concerns)
+	@echo "Running project-wide CI tests..."
+	@echo "1. Global concerns (syntax, structure, Makefile)..."
+	tests/test-ci.sh
+	@echo "2. Infrastructure layer tests..."
+	@make infra-test-ci
+	@echo "3. Application layer tests..."
+	@make app-test-ci
+	@echo "✅ All CI tests passed!"
+	
 test-unit: ## Run unit tests (configuration, scripts, syntax)
 	@echo "Running unit tests..."
 	@echo "1. Configuration and syntax validation..."
@@ -228,3 +245,4 @@ clean: ## Clean up temporary files and caches
 	@rm -rf $(TERRAFORM_DIR)/.terraform
 	@rm -f $(TERRAFORM_DIR)/terraform.tfstate.backup
 	@echo "Clean completed"
+

application/tests/test-ci.sh

@@ -0,0 +1,72 @@
+#!/bin/bash
+# Application-only CI tests - Run application-specific tests that work in GitHub runners
+# Focus: Docker Compose validation, application configuration, deployment scripts
+# Scope: No virtualization, no global repo concerns, application layer only
+
+set -euo pipefail
+
+# Configuration
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+APPLICATION_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+TEST_LOG_FILE="/tmp/torrust-application-ci-test.log"
+
+# Source shared shell utilities
+# shellcheck source=../../scripts/shell-utils.sh
+source "${PROJECT_ROOT}/scripts/shell-utils.sh"
+
+# Set log file for tee output
+export SHELL_UTILS_LOG_FILE="${TEST_LOG_FILE}"
+
+# Initialize test log
+init_test_log() {
+    init_log_file "${TEST_LOG_FILE}" "Application-only CI Tests"
+    log_info "Environment: CI (no deployment)"
+    log_info "Scope: Application layer only"
+    log_info "Application Root: ${APPLICATION_ROOT}"
+}
+
+# Test execution summary
+show_test_summary() {
+    local start_time=$1
+    local end_time
+    local duration
+    end_time=$(date +%s)
+    duration=$((end_time - start_time))
+
+    log_section "APPLICATION CI TEST SUMMARY"
+    log_info "Application CI tests completed in ${duration} seconds"
+    log_success "All application-specific tests passed!"
+    log ""
+    log_info "Note: This only validates application layer concerns."
+    log_info "Run 'make test-ci' for complete project validation."
+    log ""
+    log_info "Test log saved to: ${TEST_LOG_FILE}"
+}
+
+# Main test execution
+main() {
+    local test_start_time
+    test_start_time=$(date +%s)
+
+    init_test_log
+
+    log_section "APPLICATION-ONLY CI TESTS"
+    log_info "Running application-specific tests for GitHub runners"
+
+    cd "${PROJECT_ROOT}"
+
+    # Test 1: Application unit tests
+    log_section "TEST 1: APPLICATION CONFIGURATION & CONTAINERS"
+    log_info "Running application unit tests..."
+    if ! "${APPLICATION_ROOT}/tests/test-unit-application.sh"; then
+        log_error "Application unit tests failed"
+        exit 1
+    fi
+    log_success "Application unit tests passed"
+
+    show_test_summary "${test_start_time}"
+}
+
+# Run main function
+main "$@"

application/tests/test-unit-application.sh

@@ -59,12 +59,19 @@ test_application_config() {
 
     local failed=0
     local config_files=(
-        ".env.production"
         "compose.yaml"
     )
 
+    # Optional files (for development/local testing)
+    local optional_files=(
+        ".env"
+        ".env.production"
+        ".env.local"
+    )
+
     cd "${APPLICATION_ROOT}"
 
+    # Check required configuration files
     for config_file in "${config_files[@]}"; do
         if [[ ! -f "${config_file}" ]]; then
             log_error "Required configuration file missing: ${config_file}"
@@ -74,6 +81,19 @@ test_application_config() {
         fi
     done
 
+    # Check optional configuration files (info only, no failure)
+    local env_file_found=false
+    for config_file in "${optional_files[@]}"; do
+        if [[ -f "${config_file}" ]]; then
+            log_info "Found optional configuration file: ${config_file}"
+            env_file_found=true
+        fi
+    done
+
+    if [[ "$env_file_found" = false ]]; then
+        log_info "No environment files found (normal for CI, generated during deployment)"
+    fi
+
     # Test that configuration templates exist
     local template_dir="${APPLICATION_ROOT}/config/templates"
     if [[ -d "${template_dir}" ]]; then

docs/refactoring/copilot-instructions-testing-update.md

@@ -0,0 +1,45 @@
+# GitHub Copilot Instructions Update Summary
+
+## Changes Made
+
+Updated `.github/copilot-instructions.md` to include comprehensive guidance on the three-layer
+testing architecture to prevent future violations of separation of concerns.
+
+### 1. New Section: "Three-Layer Testing Architecture"
+
+Added a detailed section under "Testing Requirements" that explains:
+
+- **Critical importance** of maintaining layer separation
+- **Definition of each layer**:
+  - Project-Wide/Global Layer (`tests/` folder)
+  - Infrastructure Layer (`infrastructure/` folder)
+  - Application Layer (`application/` folder)
+- **Specific responsibilities** for each layer
+- **Clear hierarchy** showing orchestration relationships
+- **Common mistakes to avoid** with explicit examples
+- **GitHub Actions integration** guidance
+
+### 2. Enhanced AI Assistant Guidelines
+
+Added a specific "Testing Layer Separation (CRITICAL)" section in the AI assistants guidance that:
+
+- **Reinforces the architecture** with strong warnings
+- **Lists common violations** that should never happen
+- **Provides clear guidance** on proper command usage
+- **Emphasizes orchestration** through `make test-ci`
+
+## Key Benefits
+
+1. **Prevents regression** - Future contributors and AI assistants will understand the architecture
+2. **Clear guidance** - Explicit examples of what NOT to do
+3. **Proper orchestration** - Emphasizes using `make test-ci` for complete testing
+4. **Maintainable architecture** - Each layer stays focused on its responsibilities
+
+## Impact
+
+This documentation update ensures that the three-layer testing architecture violation we just
+fixed will not happen again. Contributors and AI assistants now have clear, explicit guidance
+on maintaining proper separation of concerns in the testing infrastructure.
+
+The documentation is strategically placed in both the general testing requirements and the AI
+assistant-specific guidelines to ensure maximum visibility and adherence to the architecture.