Security at OpenRelay
Running untrusted workloads on distributed consumer hardware requires serious security. Here's how we protect your data and keep your workloads isolated.
Hardware-Level Isolation
Every VM runs its own kernel, so VMs on one host never share one.
GPU Passthrough via VFIO
GPUs pass directly to your VM, so the host loads no GPU driver for them.
Encrypted Communications
Network traffic to your VM is encrypted in transit, and every host authenticates to our platform with its own certificate.
Gateway Authentication
Unless you make a VM public, reaching it requires your organization's credentials, which our gateway checks.
Automatic Failover
When nodes fail or become unresponsive, workloads are automatically migrated to healthy nodes.
Access Controls
Roles decide what each member can do, and API keys are scoped, rotatable and revocable.
Our Security Model
OpenRelay's security model is built on the principle of defense in depth. We assume that any component can be compromised and design our systems to contain potential breaches.
Workload Isolation
Unlike containerization, where workloads share the host kernel, each VM runs as its own lightweight virtual machine with:
- Dedicated guest kernel (no shared kernel attack surface)
- Isolated memory address space
- Its own isolated virtual network interface
- No direct access to host filesystem
GPU Security
GPUs are passed through to VMs using VFIO (Virtual Function I/O), which provides:
- Direct hardware access without host driver involvement
- IOMMU-enforced memory isolation
- No ability to access host memory or other VMs
- Complete GPU reset between workloads
Network Security
On the network:
- TLS on our API, inference and gateway endpoints
- Hosts need no inbound ports to serve your VMs: they connect out to our platform over authenticated, encrypted connections
Data Protection
Data in Transit
Traffic between you and your VM is encrypted in transit across the network, and our API redirects plain HTTP to HTTPS.
Data at Rest
Terminating a VM schedules its disks for deletion on the provider node.
Data Retention
A stopped VM keeps its disk until you terminate it, and a pod's volume stays until the pod is deleted.
Access Controls
- API Keys: Scoped, rotatable, revocable credentials for programmatic access. With permissions enabled, every new key has an owner and can be limited to specific permissions, models, endpoints and networks, and an expiry
- Organization Roles: Owner, Admin, Member and Viewer. With permissions enabled, also Billing, Inference and custom roles
- Provisioning Tokens: Short-lived (24-hour), revocable tokens for node registration, hashed storage
- Access Audit Log: With permissions enabled, every change to keys, members, roles and policies is recorded for 365 days
Infrastructure Security
Our control plane infrastructure is hosted on secure, SOC 2 compliant cloud providers with:
- Regular security patches and updates
- Network segmentation and firewalls
- Intrusion detection and monitoring
- Regular penetration testing
- Incident response procedures
Vulnerability Disclosure
We take security seriously and appreciate the efforts of security researchers. If you discover a security vulnerability, please report it responsibly:
We commit to acknowledging your report within 48 hours and providing regular updates on our progress. We do not pursue legal action against researchers who follow responsible disclosure practices.
Compliance
We are actively working toward industry compliance certifications:
- SOC 2 Type II: In progress
- GDPR: Compliant
- CCPA: Compliant
Questions about security?
We're happy to discuss our security practices in more detail. Reach out to our security team.
Contact security team→