Hi there folks, I’m still learning about Linux and have yet to dip my toes properly in any arch based distro. Have for the moment fallen in love with the immutable distros based on Universal Blue project. However I do want to learn about what arch has to offer to and plan on installing default arch when I have time. But have been wondering why I haven’t heard of any immutable distros from arch based distros yet.
So, am left wondering if there are talks within that Arch community of building immutable distros?
While writing this post I found a project called Arkane Linux, which seem to be very interesting. Does anyone have nay experience with it? Is there a specific reason why immutable wouldn’t be a good idea when based on Arch?
Project: https://arkanelinux.org/
There have been at least 1 PoCs for arch linux based on ostree: https://wiki.archlinux.org/title/User:M1cha/Install_Arch_Linux_inside_OSTree
In addition, VanillaOS’s ABRoot has been packaged through the AUR
SteamOS3 is immutable and arch-based. You can see a fan-recreation of the image builder here
Otherwise, you can use the alpine linux immutable root with atomic upgrades guide.
Generally speaking, though, pacman is really basic, and the majority of the atomic/immutable magic happens in the package manager. That’s why only existing, complex package managers such as rpm-ostree (which shares a code base with DNF) have full support for it.
Immutable distro is mostly a buzzword. What people want is reproducible environments, and for that you should look at NixOS.
No, just because it is reproducible doesn’t mean you are able to (re)produce something that works. With something like fedora silverblue you know that this specific composition of packages and their versions has been tested, and that all the other users run this exact composition as well.
When you roll your own composition, where you install whatever stuff, you may be the one finding out that there’s some conflict between package a version u.v.w and package b version x.y.z.
Unless you both use exactly the same config files… Which is the point of nixos… Everything is versions locked. If you have a working config you can give it to your buddy and build it and it’ll work the same way
If you have a working config, thats exactly the point. Before you built your config, you don’t know. If you deploy silverblue, you know it will work beforehand because exactly this config, including /etc, has been tested upstream before. What you are to your buddy, Fedora Atomic is to me. The difference is, it is not just one person that tested some config they decided on on their single piece of hardware, it is the effort of a full blown distro team.
Both features are important IMO, reproducibility is for being able to define certain aspects of your machine in a way that you can nuke it and, as long as you have its configuration (declarative for Nix, other implementations might have it as imperative), bring it back just how it was set up, without differences or breakages; while immutability is for being always confident that whatever* you do to your machine, you won’t be able to break it because the root, which holds the functioning core of your system, can’t be messed around with, NixOS has both I believe.
*not really “whatever”, because there are still some ways to break, but you have to be very deliberate in doing it (think
rm -rf /*), but in normal operation you won’t just somehow install something or upgrade your packages and be left with an unusable systemlol
I just started toying with Arkane Linux. It’s fairly easy enough to make your own image and they provide some simple templates you can use if you don’t want Gnome. To me, the greatest thing about Arch is the AUR and unfortunately it doesn’t support AUR packages out of the box. This might not be a problem since you could mostly get along with flatpaks or distrobox. It might be a chore for someone new to Arch to have to compile something straight from the AUR that your device needs to function, like what I’ve had to do.
Thanks for insight!
Give NixOS a shot. It’s got a learning curve that may be difficult if you’ve never read code, but it’s my preferred immutable setup.
It even has more packages than Arch.
Here’s the video that got me onto it:
Have actually tried it! And while I love the concept and how it works, its a bit too much to learn for me at the moment. It’s defo something I am going to pick up again in the future though! Also amazed me exactly how many more packages it had available than the AUR, mind blowing.
Here is an alternative Piped link(s):
https://piped.video/CwfKlX3rA6E
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
steam deck? I wonder how many full-time staff valve devotes to testing and pushing regular updates.
I think a lot of arch people want the bleeding edge updates, so it seems a lot like to go btrfs or and setup snaphots or something if they want a safety net.
Aside from what others have already mentioned, atomic distros usually come with “batteries included”, they have a desktop environment and bundled software. The goal is to have a complete setup where only the user space will need to be modified (for example by installing applications through Flatpak).
Arch doesn’t really have a “batteries included” default install.
I always thought immutable distros would be for servers. Am I missing the point?
But have been wondering why I haven’t heard of any immutable distros from arch based distros yet
SteamOS running on Steamdeck is Arch and immutabl/atomic for anyone not familiar.
Another one is blendOS
blendOS keeps everything simple, delivering on application and game compatibility from various sources while offering a lightweight atomic & declarative Arch system.
astOS and Arkane Linux too.
But have been wondering why I haven’t heard of any immutable distros from arch based distros yet.
If your question is “Why doesn’t Arch have its own atomic/immutable spin/flavor like Fedora and openSUSE have in their Silverblue/Kinoite and Aeon/Kalpa respectively?”, then the answer simply lies in the fact that Fedora and openSUSE have a lot more incentive for venturing the unexplored waters of atomicity/immutability as their enterprise counterparts exist and will benefit majorly from it.
The enterprise counterparts also allow funding that is essential for erecting this from the ground. But, even then, the shift towards atomic/immutable is a difficult one with a lot of hardships and complexity. From the ones that have developed their atomic/immutable projects retroactively (so GuixSD and NixOS don’t count as they’ve been atomic/immutable (and declarative) from inception), only Fedora’s (I’d argue) have matured sufficiently. But Fedora has been at it since at least 2017, so they’ve had a head start compared to the others.
In contrast to Debian (through Canonical), Fedora (through Red Hat) and openSUSE (through SuSE), Arch has literally no (in)direct ties to enterprise. Hence, it will only adopt an atomic/immutable variant if the incentive is high from the community or if it’s very easy and only comes with major benefits. But, as even openSUSE is currently struggling with their atomic/immutable variants, it has a long road ahead before it becomes something that can be easily adopted by Arch. Hence, don’t expect Arch’s atomic/immutable variant any time soon.
However, if any derivative suffices, then at least the likes of blendOS, ChimeraOS and even SteamOS are worth mentioning here.
Thanks very much for the detailed response - this was very insightful!
In contrast to Debian (through Canonical), Fedora (through Red Hat) and openSUSE (through SuSE), Arch has literally no (in)direct ties to enterprise.
LOL Fedora and opensuse are copying from the commercial distros, but Debian is not copying Ubuntu (literally the opposite)
Tell me you don’t understand what those distros are without telling me you don’t understand what those distros are.
Fedora and opensuse are copying from the commercial distros
How are they copying if Fedora and openSUSE Tumbleweed are upstream to RHEL and SLE respectively?
Btw, I don’t understand what your comment was set out to do. Could you elaborate?
What matters is the important stuff like deciding what package format to use, how to handle the biggest bugs, default filesystem, systemd or not, and who gets to decide all this stuff and so on. Some distros follow the company decision and some do not. Get it?
Thank you for clarifying.
I’m not very familiar with how stuff works over at (open)SuSE. However, for Fedora, we know that they’ve gone against Red Hat’s policy more than once. At the end of the day, it is
(at the very least in name)a community distro.But, I think we can at least agree on the fact that Canonical’s influence on Debian is definitely less than Red Hat’s influence on Fedora or SuSE’s influence on openSUSE.
Btw, consider conveying this better next time 😅. I think most others, like me, misunderstood you 😜.
Have a nice day!
Don’t feed the trolls :)
The biggest issue with immutable OSs is the lack of containerized apps. Most devs simply don’t distribute their apps in flatpaks etc. Install fedora atomic. Fist think I want to do is install xpipe to manage my servers. Can’t be don’t in an unprivileged flatpaks. Great layer it on.
Let’s try seafile next to sync my files and projects…the flatpak is maintained by a random volunteer and most up to date version is from a year ago. Great, layer that in as well.
Let’s install a command line tool, before it was 1 line, now it’s a whole lot of googling only to discover that the best way is probably to just have a whole other package manager like brew
The concept is great and it has lots of potential, just it will only work if devs start packaging their stuff in a format that works with the new paradigm (containers)
Not everything should be flatpak’d. In your case, xpipe (and in the future, waypipe) should always be installed in a docker container containing your normal “mutable” OS. It’s why Fedora is evaluating Ptyxis: when you open a terminal, instead of defaulting to your immutable root, it can be set up to go to a container which has your home mounted but a traditional, mounted root.
Yeah, I played with Silverblue for the first time a week or two ago, when I decided to move back to Gnome from Plasma. When I realized that I’d need to layer adw-dark to get rid of the light settings panel in Gnome Console, and then layer in aptx and ldac support, and then some drivers for hardware accel in Firefox… I came to undestand that truly approaching this as minimally layering, and instead properly relying on flatpak and toolbx/distrobox wasn’t going to work out. Instead I’m just going to get anxious every time I have to say, ‘well fuck, I guess I have to layer this too.’
That and I really don’t like the mess of a filesystem. So back to Arch, with some things learned to keep stuff I don’t like out of my base system. I can use a Bazzite-Arch container for Steam, to avoid having to enable multilib, for example. Well, if I can figure out the performance issues, anyway. And I know I’m weird, but I’d kind of like to avoid using AUR on my base system, and Flatpak kind of terrifies me for the reasons you mentioned
I do look forward to an immutable future, but I don’t think it’s going to make me happy for some time. Maybe Nix or GUIX, but that sounds like a winter project. I know some folks use an Arch base with Nix layered on top, but that rather sounds like the inverse of what I’d ideally want. It seems like the beauty of Nix is that you don’t have to worry about layering, because YOU declare the base?
Can’t those be installed in toolbox?
I don’t think xpipe would work, it needs too many permissions.
Something like seafile would work, better than overlaying it I guess but still isn’t park of a package manager with easy auto updates etc like it would be if the devs published to flatpak.
At the end of the day it’s a lot more work that the promise of opening discover, searching an app and hitting install.
I know
ssh -Xworks fine in a rootless podman container, and so does waypipe. I’d be shocked if xpipe didn’t.
The biggest issue with immutable OSs is the lack of containerized apps.
Disagree. This is a non-issue for NixOS and Guix System. If anything, what you say only (somewhat) applies to Fedora Atomic or otherwise immature and/or niche immutable distributions.
For Fedora Atomic (and others that operate similarly), pet containers (read: Toolbx (and later Distrobox)) were originally envisioned as the solution. But, even Nix (and as you’ve noted brew on opinionated uBlue) has been used to that effect.
Though, yes, I don’t ignore that sometimes you just gotta layer it. Thankfully, as that’s exactly why we got that feature 😉.
What are “containerized apps”? Do they run in docker, podman, firejail, or bubblewrap? Also, what is their benefit?
Actually yes. Fedora atomic has a system called toolbox that uses podman to encapsulate desktop apps. Flatpak also provides a sandboxed container.
The idea is to keep the OS and apps separate as much as possible for both security and stability.
Honestly, I’d rather it didn’t, or at least maintain the standard Arch as it is today. I get the appeal of immutable distros/installs, but they are just not for me. They feel like a lot more work, for not much benefit.
Also, given that Fedora atomic distros are the ones people seem to compare against, I don’t like that flatpak is effectively required. I like flatpak and what it offers, I don’t like that all apps I install must be flatpak, or that I have to go around that default in some way (when I tried one of the atomic spins, I kept running into apps that just didn’t distribute as flatpak and it made things more difficult). I suppose this old dog doesn’t want to deal with some new trick just to use his system like he always has with a simple
pacman -Sorapt install.Its honestly not that bad installing apps on these distros. Yeah the main one is flatpak, secondly appimages as easy as ever, and any other app can be installed via distrobox and exported if needed. If not that, then apps can also be layered onto the system if needed.
Ultimately it boils down to user preferences, but considering the growth and popularity of Immutable versions of distros, I would be surprised to not see a few more attempts from the Arch side of Linux.
I’m not trying to be a Luddite here, I love technology when it solves a real problem. I guess I just don’t have a problem that an immutable distro would solve.
You don’t have to install everything as a flatpak if you don’t want to. You can totally install most things in a rootless distrobox container, then use
distrobox export(if you’re using distrobox instead of toolbx) to get a nice desktop entry. It’s how I run VSCode and Quartus Prime, for example.Or I could run
pacman -S codeon a system that doesn’t require hoops to jump through.I think I just don’t see the reason or benefit of going the immutable distro route. At least not yet, for me. I’ll never say never, of course. Right now it feels like extra steps to achieve the same thing.
What is the benefit of an immutable distro?
I could see it being useful for like an office or something, where you do a big roll-out to a bunch of people. I’d assume having the system files be read-only and (presumably) the same on every system would eliminate a lot of guesswork for IT troubleshooting.
For me:
- atomic updates
- reproducibility
- (to some degree) declarative system configuration
- increased security
- built-in rollback functionality
and their consequences;
- rock solid system even with relatively up to date packages
- possibility to enable automatic updates in background without fearing breakage
- (quasi) factory reset feature
- setting up a new system in just a fraction of the time required otherwise
are the primary reasons why I absolutely adore atomic/immutable distros.
Furthermore, it minimizes all kinds of issues related to or caused by bit rot, configuration drift and hidden/unknown states. (Note that you won’t reap all of these benefits on all atomic/immutable distros.)
(Note that you won’t reap all of these benefits on all atomic/immutable distros.)
That’s because most of these benefits are not a result of a distro being immutable.
You should define what “being immutable” means (according to you).
Besides, the questioner asked what the benefits of an immutable distro are. The only three mature immutable distros possess all of these qualities. And even if some of these qualities may be found on other distros that are not qualified as immutable. Fact of the matter is that the immutable variants of these features are far and wide superior over their counterparts found on traditional distros.
I guess I’d define it as a distro where the base system is read-only and changes or updates to it are done by replacing it atomically.
Fact of the matter is that the immutable variants of these features are far and wide superior over their counterparts found on traditional distros.
How exactly? Just saying it doesn’t make it true. Except for atomic updates (which are basically the main point of these distros, and why they’re also called “atomic”), what can they do that you can’t on a normal distro?
Thank you for your reply!
I guess I’d define it as a distro where the base system is read-only and changes or updates to it are done by replacing it atomically.
Aight. I got no qualms with that definition for an immutable distro. However, small nitpick, the term “base system” can be very murky at times. And perhaps I would rephrase the part addressing changes/updates to “changes or updates to it are intended to be applied atomically”.
Btw, I think this conversation is primarily on semantics and some assumptions we’re making related to that. So, I agree with you that (strictly speaking) immutability is only part of the puzzle (perhaps I might even refer to it as an enabler) for acquiring a lot of the aforementioned benefits to the degree by which it’s attained. So, the precise implementation of immutability is at least as important.
For example, openSUSE Aeon/Kalpa, as much as I like them, have not been able to deliver most of these benefits beyond what traditional distros are capable of. Despite these distros being immutable*. However, they’ve recognized their faults and intend to move towards an image-based solution in order to improve. Similarly, Vanilla OS has recognized that their first vision of ABRoot wasn’t fit and thus overhauled it to be more in line with Fedora Atomic. We should continue to regard their initial visions as immutable distros despite ‘their failings’, but should also recognize that their failures aren’t representative of what immutable distros are or can be.
How exactly?
what can they do that you can’t on a normal distro?
Alright, let’s start:
(Note that the immutable distros will only be represented by Fedora Atomic, GuixSD and NixOS. The others are either too niche or immature)
- atomic updates; no need to explain as you’ve established this yourself.
- reproducibility; I’m not talking here simply about “reproducible builds”. Instead, I’m referring to how, on a daily basis, the system is actually built up from the ground (or at least capable of doing so without the hassle of actually committing to a complete reinstall). Some references in case you’re interested to know more about this. This is something I’ve simply never seen on a traditional distro. Though I hope you can correct me on this.
- declarative system configuration; atomicity is just a pre-cursor or enabler for this. Nonetheless, ultimately, it’s a feature found exclusively on immutable distros. In case you mention Ansible, you should be aware that Ansible (at best) offers convergent system management. The aforementioned immutables succeeds effortlessly in (the superior) congruent system management. Refer to this article in case you’re interested in the differences.
- increased security; Linux’ security standards leaves a lot to be desired. Unfortunately, whatever immutability offers in that regard, is far from enough to bridge the gap. By contrast, Qubes OS, while not being immutable (and technically not even a Linux distro), succeeds at setting an excellent security standard. But even they acknowledge the benefits that immutability provides for security (even on Qubes OS).
- built-in rollback functionality; I can’t remember how GuixSD and NixOS far surpass anything else out there in this regard. However, for Fedora Atomic’s OCI model, it’s required that some registry builds your images. Currently, these images are stored for 90 days by default. Which means that I can go back to any one of these without keeping the snapshots on my own device. While it’s technically possible to keep snapshots from the last 90 days with Snapper/Timeshift, most sane people would simply not do it as it’s overkill and/or wasteful of storage. However, Fedora Atomic allows such functionality without burdening your system. Hence, it’s superior. Though, once again, perhaps this functionality is somehow available on traditional distros. But I’m simply unaware of it*.
There’s no need to go over the “consequences” as they’re (as the name implies) consequences of what has mentioned earlier. Hence, as their causes are better than the one found on traditional distros, so are the consequences better than how they’re found on traditional distros.
Finally, minimizing bit rot, configuration drift and hidden/unknown states are direct consequences of atomicity and declarative system management. Hence, immutable distros perform better at this compared to traditional distros.
(Note that the immutable distros will only be represented by Fedora, GuixSD and NixOS. The others are either too niche or immature)
This was my issue with your original comment - I’m aware most of the work on features like these is based on immutable distros, but just being immutable doesn’t mean it will have those features.
When it comes to reproducibility and declarative system management, I think you’re right that they’re only available in immutable distros.
The security benefit of a read-only filesystem isn’t very significant IMO, and for some immutable distros, interesting parts (to attackers, like /etc for example) are mutable anyway.
And I don’t use any snapshot solution currently, but don’t most of them only store the parts that change between snapshots? According to the Arch Wiki, Snapper’s “default settings will keep 10 hourly, 10 daily, 10 monthly and 10 yearly snapshots”. This doesn’t seem like much of an advantage for immutable distros, really.
There’s no need to go over the “consequences” as they’re (as the name implies) consequences of what has mentioned earlier. Hence, as their causes are better than the one found on traditional distros, so are the consequences better than how they’re found on traditional distros.
I disagree with this though. “Better” is very subjective - I for one consider being able to have an up to date system that can have parts of it updated without rebooting to be much nicer than using something like rpm-ostree, even if it is safer to use in theory (I can’t remember the last time I had an issue when installing a package; rebooting to apply an install atomically will likely make no difference to me other than wasting my time). I know I can use containers to get around this, but once again, this just adds to the hassle.
You get all of this by using Btrfs in a regular distro.
Recently kdeconnect broke on me, I just rolled back the snapshot to the day before.
You get all of this by using Btrfs in a regular distro.
No you don’t. Refer to this reply I’ve written to someone else.
Btw, Btrfs is only a file system, snapshot-functionality isn’t automatically implied with it. See traditional Fedora as a reference; i.e. defaults to Btrfs, but doesn’t set up Snapper/Timeshift or anything to that effect.
But, even then, snapshot-functionality provides only of a small subset of the benefits in an inferior way (as I’ve explained in the reply to the other person).
What do you mean by declarative system configuration? that thing that nixos does that you set it up thru its config file?
I’ve also kept several month old btrfs snapshots on my system and I don’t see a problem with it, they only add like 3 GIB of storage each when they are that old.
Also I’m not sure what you meant by increased security? Is it more secure simply because you can’t edit the root filesystem?
Yep, also ability to rebase to some other image. Maybe that’s what you meant by setting up a new system.
Rebasing is (strictly speaking) found exclusively on Fedora Atomic (though I wouldn’t be surprised if Vanilla OS has also started supporting this like Fedora Atomic does). While achieving something similar on NixOS or GuixSD isn’t necessarily hard, the term “rebase” is not used for either of these systems.
Setting up a new system with little to no nuisance is a direct consequence of managing your system declaratively. So no, I didn’t mean rebasing. Though, in your defense, Fedora Atomic does achieve it through rebasing. But, even then, it’s only one part of the puzzle.
Oh no… what is rebasing in this context? This isn’t something related to git, I imagine?
ostree is based on OCI images, the basis for containers and the like. “Rebasing” just refers to swapping out the OCI image containing your root with another.
Essentially: read-only system files.
In immutable distros, you or any other programs that are installed on the system cannot modify the system files. That includes the system configuration files as well as applications. Its goal is to solve the problem of an entity gaining admin privlieges to your system and cause loads of damage. There are some addtional benefits too:
- Updates apply at reboot
- Root partition is read-only
- Considered very secure
- Sandboxed applications via flatpaks, snaps and appimages.
But then you also can’t make any changes to the system files. I thought the point of Linux was having more control
It all boils down to user preferences right. Some users prefer the maxium amount of control, while others, including myself, only use the pc for gaming and browsing, so I’d rather have a system that cannot be broken by myself and not deal with updates etc…
The entity gaining access to system files and doing damage, it’s me.
Config files are still editable. Most of them (rpm-ostree, for example) have a mechanism for managing packages, and subsequently rolling back if anything goes wrong or completely resetting, and leave /usr/local writable. For stuff like development and working with compiler toolchains, you should be using a container. I use vscode exported in a distrobox running Fedora 40, for example.
Honestly, IMO the end-user benefit is mostly that it sounds cool.
All the benefits I’ve heard (including the ones in this discussion) don’t actually derive from “immutability” but from releases that stay the same for longer (which is what “more stable” used to mean), or the ability to roll back your system to some “known” working state (which you can do with snapshots and in a plethora of other ways).
What immutability means is that users are unable to alter their system, or at least not expected to… basically, it means what in corporate lingo would sound “altering your system is not supported” and that the distro actively makes it hard for you to do so.
This means users will not break their system because they followed badly some instructions they found on some badly written forum post anymore and blame the distro for it, but it also means that users who actually have a reason to alter their system and know what they are doing will have a hard time doing it (or be unable to), which is precisely why I left macos and went back to linux for my work computer some ten years ago (I spent half a day doing something I could have be done with in five minutes and said to myself “never again”).
For the team/company that builds it, an immutable distro will likely be easier to test and maintain than a “regular” one, which should then indirectly benefit the users (well… as long as the team/company interests are aligned with the users’ of course: shall windows get easier for microsoft to maintain, how much benefit would trickle down to its end users?).
Users who switch to an immutable distro should see a decrease in bugs short-term. In the longer run, I’d expect distros (especially the “commercial” ones) to reduce the effort they spend in QA until quality drops again to whatever level is deemed appropriate (if bread costs less I’m still not gonna buy more bread than I need… same goes for quality).
Basically, it all boils down to “immutable distros cost less to maintain” (which, don’t get me wrong, is a net positive).
I must say I find it slightly concerning to have heard several “veteran” linux users say that immutable distros are so great that they will install one on their parent/child/SO/friend’s PC but on their own.
It’s also a bit unnerving to notice that most of the push for immutability seems to come from companies (the likes of debian/arch/gentoo/etc. are not pushing for immutability AFAIK, and they certainly don’t have the initiative in this field).
I’m not sure how much immutable distros will benefit the community at large, and… I’m not even sure they will end up being very successful (windows/macos follow in whatever makes is more profitable for microsoft/apple, linux users have choice).
I hope that immutable distros will prove both successful and good for the user community at large.
edit: Forgot to explain the positives I hope for: since immutable distros should require less effort, I hope this will lead to more/better “niche” distros from small teams, and to distros with bigger teams doing more cool stuff with the extra manpower
Pretty much zero for most users
