EDDIE (Event-Driven Detector for IoT and Edge) is an infrastructure-as-code solution for deploying a complete edge computing stack on IoT gateways and edge devices. It automates the provisioning and configuration of containerized ML inference pipelines, data management services, and workflow orchestration tools optimized for resource-constrained edge environments.
EDDIE provides a turnkey solution for deploying production-ready machine learning inference at the edge. It combines lightweight Kubernetes (K3s), workflow orchestration (Argo), object storage (MinIO), IoT connectivity (Thingsboard), and integration with CyVerse data infrastructure to enable autonomous, event-driven processing of sensor data.
- Automated Infrastructure Deployment: Terraform-based provisioning with Ansible configuration management
- Edge-Optimized Architecture: K3s, MinIO, and containerized services designed for resource-constrained devices
- ML Model Management: DVC integration for versioned model deployment and updates
- IoT Platform Integration: Optional Thingsboard connectivity for telemetry and device management
- Cloud Storage Integration: CyVerse Data Store for cloud storage
- Workflow Orchestration: Argo Workflows for event-driven data processing pipelines
The deployed stack includes:
| Component | Purpose | Version |
|---|---|---|
| Docker | Container runtime | Latest |
| K3s | Lightweight Kubernetes distribution | v1.30+ |
| Argo Workflows | Workflow orchestration engine | v3.4.8+ |
| MinIO | S3-compatible object storage | Configurable |
| Thingsboard | IoT platform for telemetry (optional) | Latest |
| rclone | Cloud storage synchronization | Latest |
| DVC | Data and model version control | Latest |
- Terraform >= 1.0
- Ansible >= 2.10
- GoCommands (gocmd) - CyVerse iRODS client
- SSH access to target edge device(s)
- CyVerse account with configured data store access
- Operating System: Ubuntu 20.04+ or Debian-based Linux
- Architecture: x86_64 or ARM64
- Memory: Minimum 4GB RAM (8GB+ recommended)
- Storage: Minimum 50GB available disk space
- Network: Internet connectivity for package installation and data synchronization
- SSH: Enabled with key-based authentication
If your target IoT gateway or edge device has public network accessibility, you can use CACAO (CyVerse's cloud deployment tool) for simplified deployment.
git clone https://github.com/cyverse/eddie.git
cd eddieCreate a terraform.tfvars file in the project root:
cyverse_user = "your_cyverse_username"
cyverse_pass = "your_cyverse_password"
cyverse_asset_config_dir = "/iplant/home/your_username/path/to/configs"
instance_name = "my-edge-device"Security Note: Ensure terraform.tfvars is excluded from version control. It is included in .gitignore by default.
The cyverse_asset_config_dir should contain:
- Ansible inventory file:
<instance_name>.yaml(see Configuration section) - SSH private key:
ssh_key(permissions: 0400)
Refer to README.example-config-file for a complete configuration template.
terraform initThis will download the required Terraform providers and initialize the working directory.
terraform planReview the planned changes to ensure they match your expectations.
terraform applyOr for automated deployment:
terraform apply -auto-approveThe deployment process will:
- Download configuration files from CyVerse
- Install Ansible Galaxy role dependencies
- Execute the Ansible playbook to configure your edge device
- Deploy all services to the K3s cluster
Deployment Time: Approximately 15-30 minutes depending on network speed and device specifications.
The inventory file (configs/<instance_name>.yaml) defines your edge device and service configurations:
all:
hosts:
your-edge-device-hostname:
ansible_user: ubuntu
ansible_port: 22
vars:
# Device Configuration
device_name: "my-edge-device"
# CyVerse Integration
cyverse_user: "your_username"
cyverse_pass: "your_password"
cyverse_upload_dir: "/iplant/home/your_username/processed-data"
# MinIO Object Storage
minio_version: "RELEASE.2024-07-16T23-46-41Z"
minio_storage_size: 200 # GB
minio_bucket_name: "sensor-data"
minio_access_key: "your-minio-access-key"
minio_secret_key: "your-minio-secret-key"
# ML Model Configuration (DVC)
models_path: "/opt/models"
dvc_git_models_url: "github.com/your-org/models-repo.git"
dvc_git_token: "your-github-token"
dvc_model_path: "models/detector.pt"
dvc_branch: "main"
# Thingsboard IoT Platform (Optional)
thingsboard_host: "thingsboard.example.com"
thingsboard_device_key: "your-device-key"
thingsboard_device_secret: "your-device-secret"
# Container Images
ULTRALYTICS_IMAGE: "ultralytics/ultralytics:8.1.14"See README.example-config-file for all available configuration options.
To deploy a specific model version or branch:
model_path_override = "models/custom-detector.pt"
model_version_override = "v2.1.0" # Git tag, branch, or commit SHAFor integration with CyVerse workbench tools:
project = "My Edge AI Project"
username = "your_username"After deployment, services are accessible via kubectl or Traefik ingress:
# SSH to your edge device
ssh -i configs/ssh_key ubuntu@your-edge-device
# Check cluster status
kubectl get pods -A
# Access MinIO console
kubectl port-forward svc/minio-service 9001:9001 -n default
# Navigate to http://localhost:9001
# View Argo workflows
kubectl port-forward svc/argo-server 2746:2746 -n argo
# Navigate to http://localhost:2746EDDIE uses Argo Workflows for orchestrating ML inference pipelines. Example workflows are deployed automatically and can be customized for your use case.
# List workflows
argo list -n argo
# Submit a new workflow
argo submit /path/to/workflow.yaml -n argoUploading Data to Edge Storage:
# Using MinIO client (mc)
mc alias set edgeminio http://<edge-device-ip>:9001 <access-key> <secret-key>
mc cp local-file.jpg edgeminio/sensor-data/Synchronizing with CyVerse:
Data is automatically synchronized to CyVerse using rclone based on your configured cyverse_upload_dir.
Problem: Cannot connect to edge device
# Verify SSH connectivity
ssh -i configs/ssh_key -vvv ubuntu@your-edge-device
# Check SSH key permissions
ls -l configs/ssh_key # Should show -r--------Problem: Ansible playbook fails during deployment
# Re-run with verbose output
cd ansible
ansible-playbook -i configs/<instance>.yaml --private-key=configs/ssh_key -vvv playbook.yaml# Check K3s status
systemctl status k3s
# Check all pods
kubectl get pods --all-namespaces
# View pod logs
kubectl logs <pod-name> -n <namespace>Verify GoCommands configuration:
cd ansible
cat gocmd.yaml # Check credentials
gocmd -c gocmd.yaml ls /iplant/home/<username>eddie/
├── ansible/ # Ansible automation
│ ├── playbook.yaml # Main playbook
│ ├── requirements.yaml # Ansible Galaxy dependencies
│ ├── roles/ # Service-specific roles
│ │ ├── argo/ # Argo Workflows deployment
│ │ ├── eddie/ # EDDIE core components
│ │ ├── k8s_tools/ # Kubernetes tooling
│ │ ├── minio/ # MinIO object storage
│ │ ├── rclone/ # Cloud sync configuration
│ │ ├── thingsboard/ # Thingsboard IoT platform
│ │ └── traefik/ # Traefik ingress controller
│ └── init.sh # Initialization script
├── .cacao/ # CACAO deployment metadata
├── main.tf # Terraform main configuration
├── inputs.tf # Terraform input variables
├── terraform.tfvars-example # Example Terraform variables
├── README.example-config-file # Example Ansible inventory
└── LICENSE # BSD 3-Clause License
Contributions are welcome! Please follow these guidelines:
- Fork the repository
- Create a feature branch (
git checkout -b feature/your-feature) - Commit your changes with descriptive messages
- Test your changes thoroughly
- Submit a pull request
For bug reports and feature requests, please use the GitHub Issues page.
This project is licensed under the BSD 3-Clause License - see the LICENSE file for details.
This project is funded by:
- CyVerse - Cyberinfrastructure for the Life Sciences, [https://www.cyverse.org]
- NSF COALESCE - COntext-Aware LEarning for Sustainable CybEr-agricultural (COALESCE) systems [https://sites.google.com/view/coalescepreview]
For questions and support:
- Documentation: CyVerse Learning Center
- Issues: GitHub Issues
- Community: CyVerse User Forum
If you use EDDIE in your research, please cite:
@software{eddie2024,
title = {EDDIE: Event-Driven Detector for IoT and Edge},
author = {CyVerse},
year = {2024},
url = {https://github.com/cyverse/eddie},
license = {BSD-3-Clause}
}