Technologies

Cloudflare

Cloudflare is a popular tool for securing access to hosted resources. Cloudflare sits between DNS resolution and the web server(s), preventing an array of web-based security threats, especially denial of service (dos) attacks. Cloudflare also increases website speed by distributing static assets across its own content delivery network (cdn).

ESET

Antivirus is crucial for any public facing service. Beyond the layers of firewalls and other security measures employed by Web Infrastructure, server filesystems are also regularly scanned and protected by ESET's server security.

Google Workspace

It's Google. While Web Infrastructure doesn't use the Google Cloud Platform, it does use Google Workspace for managing and securing email. Domain Key Identified Mail (DKIM) and the Sender Policy Framework (SPF) are provided by Google Workspace the Gmail API. Mail is sent via Gmail's Simple Mail Transfer Protocol (SMTP) servers.

Google is also used for analytics and off-site storage of service data.

Kubernetes

Kubernetes provides scheduling and server management: allowing hardware to be scaled independently of hosted services.

Landscape

Updates can be hard to manage at scale. Luckily, Canonical's Landscape makes server updates and management relatively trivial and can be deployed entirely on premises.

MetalLB

Services can be reliably addressed within the globally distributed Kubernetes cluster through MetalLB provided ip addresses. This abstracts the cluster provided resources away from the hosting mechanisms making them easier to consume.

Rclone

Rclone is a magic software that allows accessing remote cloud storage as if it were local. Rclone is used at the server level for moving snapshots of raw files to remote data centers (i.e. Google).

Syncthing

The data required to run a web service, such as files and databases, must be replicated across all servers capable of hosting that service. Presently, Syncthing is used for this replication. Syncthing allows encrypted, peer to peer (p2p) data synchronization and doesn't leak data to any external company. Beyond that, it plays very nicely with the other tools used in this architecture. Unfortunately, Syncthing requires full file replication, instead of sharding data, and can cause services to break when hosted by multiple services. It can also, occasionally, break services by not fully replicating metadata, such as proper ownership permissions. Because of these flaws, a better suited solution for data replication within the Web Infrastructure is being developed; that project is the Cache Tier File System.

Tinc

To ensure safe communication between servers over the open internet, software communication is encapsulated within a Tinc mesh virtual private network (vpn). As a mesh vpn, servers communicate directly with each other when possible and route traffic through other servers on the network when it is not. This also means that any server can fail and traffic will be routed around the failed server automatically, preventing any downtime in communication.

Traefik

Ingress routing across the globally distributed cloud is reliably handled by Traefik. Traefik runs as a daemon set on all service nodes and is exposed as a load balancer within the cluster vpn. This allows additional firewall servers to bridge the gap between the public network and the cluster by forwarding all incoming traffic to Traefik.