i3D.net logo in blue
i3D.net logo in blue

Blog

›

Hybrid K8s deployment: Cloud flexibility with bare metal performance

Hybrid K8s deployment: Cloud flexibility with bare metal performance

Kubernetes Sidero i3dnet
Jaunius Satkauskas - Product Marketing Manager at i3D.net
Jaunius Satkauskas
Product Marketing Manager
Linkedin
7 June 2024

This article delves into the benefits of a hybrid K8s environment, offering insights into optimizing your setup for peak efficiency. In the quest for efficiency and cost savings, the hybrid Kubernetes (K8s) setup is a game-changer. Combining cloud-based control planes with bare metal servers for worker nodes enhances performance, reduces latency and broadens geographical coverage. This approach not only slashes costs but also boosts infrastructure reliability and scalability.

Table of Contents

Running your K8s control plane in the cloud and worker nodes on bare metal

Why does Hybrid Kubernetes matter?

Picture this: You’ve built your K8s setup on the cloud. You leveraged all the additional services to set it up and are ready for production or already are in production — but then, you run a cost audit and you realize the power of the cloud comes with a price.

Benefits of the cloud

The cloud has many positives — additional services that help you manage the control plane, elastic provisioning to meet dynamic demand and others as well. However, it comes at a cost, both financial and operational. This cost may be warranted for some clusters, or even some nodes of a cluster — but may not be necessary for things such as relatively static workloads.

Bare metal brings performance and cost efficiency

This is where Bare Metal platforms truly shine. Bare metal is cheaper, more performance-focused, often with a specialty or niche (low latency, GPUs, geographic coverage) that can benefit you, and perfectly capable of running the worker nodes. Hybrid Kubernetes lets you have the best of both worlds — running workers on powerful, cost-effective bare metal, the control plane nodes in the cloud, and also takes advantage of the cloud for extra worker capacity on-demand, if needed.

Contingencies of running Hybrid Kubernetes

While the cloud providers do offer solutions (Google Anthos, Amazon EKS anywhere, Azure AKS) for running the same Kubernetes on bare metal that you can run in their cloud environments, these solutions are quite constrained.

Going truly hybrid

Not surprisingly, all the solutions interoperate only with their own cloud. More surprisingly, none of the cloud provider’s solutions support hybrid clusters that span their cloud and bare metal servers within the same cluster. The status quo is that clusters must be comprised of all cloud nodes, or all bare metal nodes.

The advantages of true hybrid Kubernetes clusters that can span bare metal and cloud nodes are substantial.

Bare metal for compute-intensive workloads

Cloud providers often offer a large variety of machines. This allows taking advantage of appropriately sized machines for each workload (for example, small, cheaper nodes may be perfect for the control plane for a small, or relatively static cluster, or for specific workloads such as image registries). Powerful bare metal nodes can be used for compute-intensive workers, but there may not be small nodes available in the bare metal environment, and it would not be cost-effective to use an oversized node for a control plane.

Flexibility and simplicity help in the long run

Not being tied to a specific cloud provider increases flexibility in terms of where nodes in a cluster are physically located. Workers or control plane nodes can be deployed in the cloud to supplement the bare metal nodes, and located for the optimal latency or cost profile, even if this encompasses multiple cloud providers in the same cluster.

Provisioning a single cluster or a small number of clusters that span environments can result in significantly simpler maintenance and ongoing operational costs, as compared to creating clusters for each environment (Fewer clusters to upgrade, configure and secure). 

A one-size-fits-all solution is never the best idea

There are, of course, cases where a single hybrid cluster is the wrong choice. Spanning a cluster across environments increases the chances that a set of workers will temporarily be unable to reach the Kubernetes API endpoint. This can lead to ingress controllers going down, with the Core-DNS being unable to refresh cached service mappings, and new pods would be unable to be scheduled on the disconnected nodes. Thus, whether a hybrid cluster is a correct choice depends on the workloads deployed — for some applications, such as gaming, where the workers generally run a small number of self-sufficient containers, it is often a good choice. And this is where Talos Linux ​from Sidero Labs ​comes in. Talos is a Linux distribution designed for Kubernetes, with functionality called KubePrism designed to minimize Kubernetes API endpoint reachability issues, by routing such requests not just to the API endpoint, but also to all control plane nodes directly.

i3D.net and Sidero

i3D.net has partnered with Sidero Labs to offer the most flexible Kubernetes service that is optimized for game hosting.

Tailor-made for Kubernetes

Sidero Labs are the creators of Talos Linux, which, as noted above, is a Linux distribution designed from scratch for Kubernetes. It consists of only 12 binaries — just enough to manage the filesystem, networking, and load Kubernetes. Most have been written from scratch by the Sidero Labs team, in memory-safe Go. It is not derived from a traditional Linux distribution — it is in fact very untraditional – there is no systemd, bash, ssh or even shell.

This results in some important benefits:

  • Talos Linux is very lightweight, leaving almost all the resources to Kubernetes. 
  • The attack surface is very small, due to the small number of binaries, and improved by the fact that these are not standard Linux binaries, but ones written in Go. 
  • The entire operating system is designed for immutability and security: file systems are immutable, loadable modules are disabled, etc. 
  • ​​These characteristics​​ ​result​ ​in extremely fast boot times, meaning cluster scaling happens extremely quickly. 
  • The management of Talos Linux is not done by SSH or command line tools, but by an integrated gRPC secured API. 
  • Robust wireguard encryption is built in and ​is simple to enable ​to intelligently encrypt all node-to-node traffic, thus ​enabling​ clusters to span environments, securely.

 

Importantly, Talos Linux can be run anywhere Linux can run — on all major public clouds, VMware, bare metal, even SBCs — and ​is​​ ​managed the same, and deploys the same vanilla Kubernetes​, in all environments​.

Omni brings more benefits

Sidero Labs has also created Omni, a SaaS service that simplifies the secure deployment and management of Kubernetes clusters, making cluster creation as easy as booting a bare metal node or cloud machine off an ISO, cloud image or disk image, and ​issuing ​a simple UI or API command. It makes the management of multiple clusters, in multiple locations (and even spanning multiple locations) extremely simple. 

Customers are taking advantage of Omni and i3D.net’s ​blazing-fast​​ ​hardware to deliver optimal performance, while also being able to leverage other data centers in the cloud for cost or latency optimization. This has proven very valuable, allowing customers to get the stability, performance and cost-effectiveness of i3D.net’s infrastructure, while also bursting to the cloud for instant capacity on-demand, to meet surges such as game launches.

Automated deployment of worker nodes on bare metal with control plane in the cloud

With Talos Linux distributions and Sidero Labs’s Omni service, automated deployment of worker nodes on bare metal with the control plane in the cloud has never been easier and offers a compelling blend of reliability, performance and scalability for Kubernetes environments. This approach allows for seamlessly transitioning workloads from on-premise infrastructure to a hybrid model that leverages bare metal’s robustness and inherent cost efficiencies together with public cloud’s flexibility. Such configuration ensures optimal coverage by combining the best of on-premise resources and cloud services, eliminating the drawbacks of shared hosting environments, like the “noisy neighbor” problem. It also presents notable cost advantages by optimizing resource allocation based on demand.

Gaming applications need performance

Particularly for gaming applications, where low latency and high performance are non-negotiable, deploying worker nodes on i3D.net’s bare metal ensures uninterrupted gaming experiences for your players due to superior network infrastructure cutting latency on every corner, as well as robust Global Low Latency Anti-DDoS (GLAD) protecting you on​ the​ bit-level. With scaling and orchestration mechanisms in place, i3D.net guarantees resource capacity to handle peak loads. Moreover, i3D.net’s architecture supports a cloud-agnostic approach, offering the freedom to choose or switch between cloud providers, whether it’s AWS, GCP, Azure or Tencent. It even allows you to bring your own cloud account for ​specific pricing​ you might have ​negotiated ​with the cloud provider. This flexibility not only ensures redundancy but also facilitates a distributed control plane, enhancing the system’s resilience and reliability throughout the Kubernetes clusters. 

Scalability is key

With multiple scalability strategies in place, i3D.net can integrate additional worker nodes seamlessly and automatically into the existing infrastructure without impacting the operational efficiency of the deployed applications. This scalable nature ensures that the infrastructure can adapt to varying workloads, making it an ideal choice for applications that experience fluctuating levels of traffic.

The automated deployment of worker nodes on bare metal, with the control plane in the cloud provides a solid foundation for deploying scalable, high-performance Kubernetes environments that are optimized for demanding applications such as online gaming.

In short

Hybrid Kubernetes setups reveal a transformative approach to infrastructure management, balancing cost efficiency, performance and scalability. By deploying the control plane in the cloud and worker nodes on bare metal, businesses can leverage the inherent strengths of both environments. This model not only offers a solution to the financial and operational challenges of cloud reliance but also enhances performance through the strategic use of bare metal servers. With the added flexibility of geographical deployment and the ability to scale with demand, the hybrid Kubernetes environment emerges as an optimal choice for organizations aiming to maximize their infrastructure’s reliability, efficiency and overall impact.

Main Take-Aways

This article assesses the benefits of a hybrid K8s environment and how it can help optimize your setup for peak efficiency. 

Related Links
View all News

Need help from our Customer Support Team?

Get Support

Sales

USA flag round icon

1-800-482-6910

NL flag round icon small

+31 (0)10 890 00 70

Subscribe to our Newsletter and stay up to date with the latest from i3D.net

Subscribe

Features

Solutions

Products

Company

© i3D.net | All rights reserved.

Twitter Linkedin Facebook-f Youtube Instagram
This site is registered on wpml.org as a development site. Switch to a production site key to remove this banner.