By project, I am talking about a “virtual” instance. I get to use a computer on their server, but how? Are those virtual machines, or containers?
If the former, then why - given how virtual machines have a large overhead? If the latter, then containers have a low degree of isolation compared to VMs, right? I’ve also heard about K3 and K8 before, but I don’t know exactly what they are, and what role they place here.
And speaking about either of them - how are they introduced through a backend - by using bindings? Or is there a port-equivalent to pass instructions, similar to how we connect to a database?
It’s very complex with hyper visors and virtualization technology. I don’t fully understand it myself in terms of how resources are allocated across something like aws or azure, but take a look at openshift vs openstack maybe. Openshift is for deploying containers and openstack is virtual machines. Openshift is kubernetes with some customizations for enterprise. Openstack is same for vm’s.
Instances are virtual machines which tend to host an operating system, and a container is lighter and only hosts an application where the code and dependencies are isolated from the underlying operating system it runs on. k8 is kubernetes, which is container orchestration. I think of virtual machines for jobs that scale vertically, while containers are suited to jobs that scale horizontally. But this isn’t necessarily true as kubernetes is starting to get slurm functionality using tools like sunk.
For integrating these things it depends on the application. You can run services in either by exposing ports and interact through API end points that point at them, eg for frontend web app serving data from a database hosted on a server or a container via fastapi. But I’m no dev ops engineer and the field is very complicated. There are many discussions around building micro services (containers) vs monolith (vm). Many decisions depend on the project. Hopefully some actual dev ops engineers will chime in and correct all of the above! xD
It’s VMs. The overhead is not nearly as bad as you think it is, especially with the highly tuned cloud hypervisors. I’ve seen dual EPYC monsters running 300+ VMs. Server CPUs are basically designed for that kind of workload these days.
Virtualization tech is really, really good. On my desktop, I have a VM that runs Windows+SteamOS with a passed through GPU, game on it and everything. You wouldn’t know it’s a VM. The overhead is so low that I just let it run in the background most of the time. When it’s idling it basically just occupies RAM. You can’t really feel the VM on the host either, everything is as responsive as usual. As long as there’s enough resources for everyone, you can barely tell it’s a VM or not.
Modern CPUs have extensions to handle it at the processor level, and most operating systems have good paravirtualized devices, so there’s not a whole lot of overhead left other than the guest kernel and processes.
Never tried passing the GPU through. You use virtualbox or something else?
On Linux KVM is what people use for this. Not an option in VirtualBox.
Using VirtualBox is also how you get the impression virtual machines are slow and heavy. It served us well but it’s essentially obsolete at this point.
The virtio devices and drivers are much better overall and optimized for performance. VirtualBox is optimized to be user friendly and run most operating systems easily. It’s compatible but to do so, it relies a lot on emulating existing hardware. Virtio devices often rely on either shared memory for zero copy, or at least DMA and exposes higher level APIs. No point encoding and decoding SATA commands when you can just move raw data in and out of the VM.
We also have hardware with native VM support, so multiple VMs can talk directly to the network without going through the host at all. VirtualBox can’t make use of those either, as it requires IOMMU and the same stuff that powers VFIO.
KVM + QEMU is where it’s at. There’s also cloud-hypervisor which is supposed to be even better for cloud use cases.
Depending on implementation, virtual machines can get to like, 99% of the performance of bare metal. So the “overhead” of virtualisation outweighs the alternative of needing 1 machine per client
edit - containers, machines, k8s etc.
On some cloud providers you can use entirely isolated machines, if that is a requirement.
If not, you use virtual machines which provide x cpu y memory etc. That VM is isolated from other VMs and the host.
Above that you can get constructs (cloud specific) like App Services (azure). Its a virtual machine but abstracted further so you are deploying into the web server on a virtual machine without needing to configure the VM or web server.
Containers are sort of like virtual machines but they are designed to isolate and run single processes.
K8s (Kubernetes) is a container orchestration platform. Its a system you deploy, and into which you can add containers, networking, that sort of thing. Kubernetes then manages the lifecycle, creation and tear down of these containers.
This a simpler overview written on my phone, someone may come add more details / correct me
