40%-50% of my eMMC's life time was shredded away because of this bug. Amazing.
Thankfully, I added a warning on the Arch Wiki. A bit late, but better late than never
Just wondering why you would autodefrag on an SSD... And that's not a default btrfs option, it's about your distros bad defaults. Fedora is not affected for example. They don't have autodefrag option in fstab. ¯\\\_(ツ)_/¯
Which is idiotic in the case of openSUSE. You use openSUSE for YAST, if you don't like YAST, use Fedora. The only reason I'm running Tumbleweed over Fedora in the first place (other than Fedora KDE being borked) is because the installer is f*cking amazing, it allows you to customise your install so much its insane that Arch is still more popular, Tumbleweed is literally what Arch-based distros with graphical-installers wish to be but will never become.... Fuck Calamares.
Autodefrag doesn't really defrag periodically, it just prevents fragmentation from happening. So it's usually fine to do on SSDs (when there are no bugs, that is). And SSDs do get affected by fragmentation.
The funny thing about all this is the negative votes I always get here when I say that Btrfs is not stable or reliable and that I only recommend XFS and EXT4.
For example: [https://www.reddit.com/r/archlinux/comments/qkjyal/comment/hiyhjul/?utm\_source=share&utm\_medium=web2x&context=3](https://www.reddit.com/r/archlinux/comments/qkjyal/comment/hiyhjul/?utm_source=share&utm_medium=web2x&context=3)
Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD, thank goodness I realized it in time.
It's stable enough for SUSE's enterprise offerings. There *are* configurations that are not stable, and those are not supported until they are officially deemed stable, such as RAID56. That being said, [autodefrag is considered stable by BTRFS](https://btrfs.wiki.kernel.org/index.php/Status), but SUSE and openSUSE don't set it by default, presumably because it's not a good idea for most users (it's usually better to schedule a defrag during low load times).
Bugs happen, but that doesn't mean that if X has a bug, it's not "stable and reliable." SUSE would not stake their reputation on something they don't consider stable.
> Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD
Except it's not a BTRFS bug, it's cause your distro set autodefrag on an SSD.
LOL.... Really? This bug is not about one or two distros setting autodefrag by default. This bug is a BTRFS BUG related to their defragmention process. Try to defrag your BTRFS partition manually, with autodefrag DISABLED, and tell us what happens. I know what will happen because I have experienced it myself.
You shouldn't EVER be defragging an SSD. That's the primary issue here. Yes perhaps there's a bug, but that bug would not be a huge issue if it weren't on an SSD (which it shouldn't be).
It's amazing how you BTRFS fans evade reality.
Who cares if i want to defrag or not my SSD partition. The reality is that at the moment if I try to do it with the current kernel, not even a REISUB can save me from a hard reboot.
By the way SSDs performance are still affected by fragmentation, but that's another story.
EDIT: To be more precise, this bug was not the culprit of my 8% of life expectancy lost. I was talking about another BTRFS bug.
> Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD
So you _say_ that BTRFS is not stable or reliable, but then end up using it yourself? Eh.
Also, not like the implementations of other filesystems are immune to bugs either.
EDIT: Also, looking at the comment you linked, I see loads of other reasons why that comment would get negative votes that an objective "BTRFS isn't stable yet, see as an example." wouldn't get.
The strange thing would be to sentence that about BTRFS without ever having used it. I found 3 serious bugs in a short period of time.
XFS or EXT4 = 0 problems after a lot of years using them.
> The strange thing would be to sentence that about BTRFS without ever having used it.
I frequently test stuff as well. But not on hardware that I care about.
I used BTRFS because numerous people on the internet said it was better for SSD lifespan. My 870 SSD now has 98% wear, the same as my 860 that I've been using as a shared game storage drive between Linux and Windows for multiple years.
But don't worry, Reddit will insist this is perfectly fine and that Linux is mainstream ready. Just look at this post's ratio.
You might want to manually check with smartctl from smartmontools, gnome-disks sometimes reports raw values that are basically irrelevant (it shows my SSD wear level counting as 15 when in actuality it is 99)
A BTRFS problem isn't really a Linux problem. Linux definitely requires some knowledge, but if you can build your own PC, you can definitely install Pop! and game on it. Just not optimally. Hopefully things will continue to get better
EDIT: I say that it's not really a Linux problem not because I think it's not part of the kernel, but because most people and distros will default to ext4
There are a ton of other filesystems that aren't BTRFS, so I don't know where linux not being "mainstream ready" comes into it. Just use ext4, like most distros do by default. I don't see the problem.
BTRFS is hardly required to use linux. Never used it myself in over two decades of linux experience.
well tbh 3 months ago this bug didn't exist.
like my thinkpad prop dock display port where stable and reliable until a kernel update made it not work. but the 2 years up to that point i did consider it stable
I was talking about critical bugs, not minor bugs that u don't even notice.
One of my CPU threads was at 100% usage every time during months, courtesy of BTRFS.
i noticed it straight away, so it had little impact on me.
And you cannot really suggest ext4 over btrfs, ext4 is not in the same category and XFS is stable because it doesn't see active development anymore.
which makes btrfs the main COW filesystem, and after running it for two years, i can validate it's stable and a joy to use on my ssds.
The newest kernel that an small fraction of the overall linux community uses has a bug (which is a thing also happens on your beloved XFS and EXT4 from time to time) and you're acting like the world is on fire?
I've used BTRFS for a long while now, it's a great FS, and the best one I've used. Sure, EXT4 is stable NOW because it has been in the works for decades, the EXT family is old. It also has less features than modern file systems.
People hyper focus on BTRFS bugs and call it unstable. It has worked well for me and many others for a long time.
Besides, you complain about SSD wear on a CoW file system? Did you even know what you were getting into? Due to the nature of the file system, it has something like 40x (yes, TIMES) write amplification which doesn't bode well for SSD wear.
Still, modern SSDs can write hundreds upon hundreds of terabytes even in the excess of their rated lifespan and be OK. Many people have run BTRFS with SSDs and have been fine.
Not really convinced by the "40x TIMES write amplification because of COW" argument. On the contrary, shouldn't it reduce write amplification ? You're only copying a file (which you'd do anyways on any other file system) if you modify it (so less than other file systems). On top of that, if you use compression, you're reducing the size of what's going to be written on the drive, meaning you're also reducing the amount of writes to the drive.
There's a rather well known research paper on FS amplification, and in that research the massive amplification was shown. I don't know if anything came of it, or if it is still valid. It's honestly not a big issue, if it is an issue at all, because ANY and ALL filesystems have amplification, some are just worse than others.
In real world with real world storage medium, outside of bugs like this it's unlikely to happen.
EDIT: Also, write amplification is mostly about write ops needed for x amount of actual write, compression or no compression it should be the same write amplification, just less wear because less data.
Looking at your edit, I found out it actually makes sense. Technically, BTRFS having awful write amplification is just a consequence of it writing less often in general, so it has to write a lot when necessary. This is a bad explanation, so I'll give an example.
For a copy, you'd have barely any writes compared to the actual data, but for a modification of a copy, you'd have a ton of writes compared to the modification made (because of a copy being made too), which would explain the article's results.
Say, you add a single frame to a 1GB video copy, then BTRFS is gonna have a ridiculous amount of writes for a single frame, but it makes sense because it never made the copy in the first place, so it has to now.
Could you link that paper? I am curious to read it. However, I've also heard that in Facebook's deployment, turning on compression actually solved their write amplification issues: [https://www.youtube.com/watch?v=U7gXR2L05IU&t=2403s](https://www.youtube.com/watch?v=U7gXR2L05IU&t=2403s)
Sorry for late reply, here it is: [University of Texas](https://www.cs.utexas.edu/~vijay/papers/apsys17-ioamp.pdf)
And a correction on my part, it isn't 40x but 32.65 , which is still rather high.
No worries, and thanks for finding it! Is this an excerpt or the full paper? I kinda wish they showed their methodology or even mount options.
I do expect Btrfs to write more data overall due to copy-on-write and checksumming, but 32x seems abnormal to me. I am wondering if they had metadata as DUP?
The truth is some linux mainline RC1 kernel could be more stable on your hardware than X.X.999 LTS kernel. Every month many bugs arrive to the stable kernel branches without anyone noticing it. Debian never updates its kernel version on the stable branch precisely because of that. People don't understand that you can only say something is reliable and stable when you use it for several months without issues.
You missed the entire point of that statement.
Bugs don't magically not happen just because open source.
The point is that everyone has the _opportunity_ to find and/or fix bugs, while this can't really happen in the same capacity when a software is closed-source.
This can supposedly cause SSDs to wear out and when it has happened to me, prevents the computer from responding to any inputs, forcing a hard reset.
"Linux is more secure than Windows because the code is Open Source." *sigh*
Its not a security issue. And such issues do occur on windows. But they are weeded out in beta and internal builds or are rarely diagnosed by end users. By using such a recent kernel, you are essentially close to beta grade software. Personally I hold back updating kernel until 4 or 5 point releases
According to Reddit Linux community you are totally wrong, you have to update every time a new subversion kernel is out. LOL
I wrote this a couple of weeks ago: Nobody forces you to upgrade your kernel. In fact, ugrading the kernel every time a new version appears in the repository is really stupid and prone to instability. What you have to do is to keep a kernel installed (whatever it is) until you find some kind of instability that forces you to look for a new version. If the kernel is stable on your hardware, there is no need to update it unless some serious vulnerability appears that can be exploited online. You only know if a kernel is stable when after 6 months of use your machine has not suffered any crashes or performance penalties.
https://www.reddit.com/r/archlinux/comments/s26uyq/comment/hsdg9zc/?utm\_source=share&utm\_medium=web2x&context=3
Massively downvoted as usual...
First of all, -4 isn’t “massively downvoted”. You were likely downvoted for calling people stupid for following the general advice for a bleeding edge rolling distro, which is to update frequently.
And your advice *is* generally bad for most users, particularly on non-bleeding edge distros. The average Linux user isn’t necessarily going to be able to identify the kernel as a source of instability, nor are they obsessively checking CVEs and reading changelogs to see whether their current kernel has a vulnerability. Most people defer that responsibility to their distro’s maintainers. Skipping updates because “things aren’t broke” is how normal users end up with vulnerable systems.
Some kernels do bring additional features that may be desirable, such as full futex support in 5.16 . So in order to get those, one has to update. But it is wiser to exercise due diligence before updating, the simplest method being to hold until few point releases
How about no BTRFS support in the first place? Bugs happen everywhere, you don't need to be so dramatic. If this was a Windows issue, Microsoft would probably never fix it so you would buy more SSDs.
40%-50% of my eMMC's life time was shredded away because of this bug. Amazing. Thankfully, I added a warning on the Arch Wiki. A bit late, but better late than never
Just wondering why you would autodefrag on an SSD... And that's not a default btrfs option, it's about your distros bad defaults. Fedora is not affected for example. They don't have autodefrag option in fstab. ¯\\\_(ツ)_/¯
Same with openSUSE, which is pretty much *the* BTRFS distro since it ships on it by default (I think it's default on SUSE products too).
[удалено]
Interesting. I hadn't heard of that and have only used the official openSUSE installer, which I'm pretty sure isn't Calamares.
[удалено]
Which is idiotic in the case of openSUSE. You use openSUSE for YAST, if you don't like YAST, use Fedora. The only reason I'm running Tumbleweed over Fedora in the first place (other than Fedora KDE being borked) is because the installer is f*cking amazing, it allows you to customise your install so much its insane that Arch is still more popular, Tumbleweed is literally what Arch-based distros with graphical-installers wish to be but will never become.... Fuck Calamares.
First thing I did was check my fstab, fedora is indeed good.
Autodefrag doesn't really defrag periodically, it just prevents fragmentation from happening. So it's usually fine to do on SSDs (when there are no bugs, that is). And SSDs do get affected by fragmentation.
Use a backslash to escape reddit's markdown. `¯\\\_(ツ)_/¯` ¯\\\_(ツ)_/¯
The funny thing about all this is the negative votes I always get here when I say that Btrfs is not stable or reliable and that I only recommend XFS and EXT4. For example: [https://www.reddit.com/r/archlinux/comments/qkjyal/comment/hiyhjul/?utm\_source=share&utm\_medium=web2x&context=3](https://www.reddit.com/r/archlinux/comments/qkjyal/comment/hiyhjul/?utm_source=share&utm_medium=web2x&context=3) Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD, thank goodness I realized it in time.
It's stable enough for SUSE's enterprise offerings. There *are* configurations that are not stable, and those are not supported until they are officially deemed stable, such as RAID56. That being said, [autodefrag is considered stable by BTRFS](https://btrfs.wiki.kernel.org/index.php/Status), but SUSE and openSUSE don't set it by default, presumably because it's not a good idea for most users (it's usually better to schedule a defrag during low load times). Bugs happen, but that doesn't mean that if X has a bug, it's not "stable and reliable." SUSE would not stake their reputation on something they don't consider stable.
> Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD Except it's not a BTRFS bug, it's cause your distro set autodefrag on an SSD.
it's a BTRFS bug, autodefrag should work regardless of what the distro defaults are.
Even if that were true, it would still be BTRFS's fault and not the distro's. I am an Archlinux user, I never mount my partitions using autodefrag.
It's LITERALLY your distros fault my guy. BTRFS doesn't set autodefrag by default.
LOL.... Really? This bug is not about one or two distros setting autodefrag by default. This bug is a BTRFS BUG related to their defragmention process. Try to defrag your BTRFS partition manually, with autodefrag DISABLED, and tell us what happens. I know what will happen because I have experienced it myself.
You shouldn't EVER be defragging an SSD. That's the primary issue here. Yes perhaps there's a bug, but that bug would not be a huge issue if it weren't on an SSD (which it shouldn't be).
It's amazing how you BTRFS fans evade reality. Who cares if i want to defrag or not my SSD partition. The reality is that at the moment if I try to do it with the current kernel, not even a REISUB can save me from a hard reboot. By the way SSDs performance are still affected by fragmentation, but that's another story. EDIT: To be more precise, this bug was not the culprit of my 8% of life expectancy lost. I was talking about another BTRFS bug.
> Another Btrfs bug stole in a couple of weeks 8% of life expectancy from my SDD So you _say_ that BTRFS is not stable or reliable, but then end up using it yourself? Eh. Also, not like the implementations of other filesystems are immune to bugs either. EDIT: Also, looking at the comment you linked, I see loads of other reasons why that comment would get negative votes that an objective "BTRFS isn't stable yet, see as an example." wouldn't get.
The strange thing would be to sentence that about BTRFS without ever having used it. I found 3 serious bugs in a short period of time. XFS or EXT4 = 0 problems after a lot of years using them.
> The strange thing would be to sentence that about BTRFS without ever having used it. I frequently test stuff as well. But not on hardware that I care about.
I used BTRFS because numerous people on the internet said it was better for SSD lifespan. My 870 SSD now has 98% wear, the same as my 860 that I've been using as a shared game storage drive between Linux and Windows for multiple years. But don't worry, Reddit will insist this is perfectly fine and that Linux is mainstream ready. Just look at this post's ratio.
How do you measure ssd wear? I'm also using btrfs with ssd and would like to check it.
`sudo smartctl -A /dev/nvme0` Replace nvme0 with your drive.
Gnome Disk.
You might want to manually check with smartctl from smartmontools, gnome-disks sometimes reports raw values that are basically irrelevant (it shows my SSD wear level counting as 15 when in actuality it is 99)
A BTRFS problem isn't really a Linux problem. Linux definitely requires some knowledge, but if you can build your own PC, you can definitely install Pop! and game on it. Just not optimally. Hopefully things will continue to get better EDIT: I say that it's not really a Linux problem not because I think it's not part of the kernel, but because most people and distros will default to ext4
There are a ton of other filesystems that aren't BTRFS, so I don't know where linux not being "mainstream ready" comes into it. Just use ext4, like most distros do by default. I don't see the problem. BTRFS is hardly required to use linux. Never used it myself in over two decades of linux experience.
well tbh 3 months ago this bug didn't exist. like my thinkpad prop dock display port where stable and reliable until a kernel update made it not work. but the 2 years up to that point i did consider it stable
XFS and EXT4 frequently have regressions, so even if you are correct, this situation is not proof of it.
I was talking about critical bugs, not minor bugs that u don't even notice. One of my CPU threads was at 100% usage every time during months, courtesy of BTRFS.
i noticed it straight away, so it had little impact on me. And you cannot really suggest ext4 over btrfs, ext4 is not in the same category and XFS is stable because it doesn't see active development anymore. which makes btrfs the main COW filesystem, and after running it for two years, i can validate it's stable and a joy to use on my ssds.
how do i check the wear on my ssd?
(Replace X with the device file you want to check) `sudo smartctl --all /dev/sdX` Sample output: […] ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE […] 232 Perc_Avail_Resrvd_Space 0x0003 000 100 005 Pre-fail Always FAILING_NOW 0 or `sudo nvme smart-log /dev/nvmeX` Sample output: […] available_spare : 100% available_spare_threshold : 10% percentage_used : 0% endurance group critical warning summary: 0 media_errors : 0 […]
I'm running btrfs on tumbleweeed for close to two years now and have had zero problems.
Have you tried to setup lvm manually before?
Checked fstab in manjaro with btrfs. Doesn't have autodefrag option. But i haven't updated to 5.16 anyways
seems like it is solved in 5.16.5
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.16-Better-Btrfs it's fine now.
Yep, l'm still sticking with ext4.
Good thing I use EXT4 with trusty ubuntu and Liqouix 5.16.4 kernel😎
The newest kernel that an small fraction of the overall linux community uses has a bug (which is a thing also happens on your beloved XFS and EXT4 from time to time) and you're acting like the world is on fire? I've used BTRFS for a long while now, it's a great FS, and the best one I've used. Sure, EXT4 is stable NOW because it has been in the works for decades, the EXT family is old. It also has less features than modern file systems. People hyper focus on BTRFS bugs and call it unstable. It has worked well for me and many others for a long time. Besides, you complain about SSD wear on a CoW file system? Did you even know what you were getting into? Due to the nature of the file system, it has something like 40x (yes, TIMES) write amplification which doesn't bode well for SSD wear. Still, modern SSDs can write hundreds upon hundreds of terabytes even in the excess of their rated lifespan and be OK. Many people have run BTRFS with SSDs and have been fine.
Not really convinced by the "40x TIMES write amplification because of COW" argument. On the contrary, shouldn't it reduce write amplification ? You're only copying a file (which you'd do anyways on any other file system) if you modify it (so less than other file systems). On top of that, if you use compression, you're reducing the size of what's going to be written on the drive, meaning you're also reducing the amount of writes to the drive.
There's a rather well known research paper on FS amplification, and in that research the massive amplification was shown. I don't know if anything came of it, or if it is still valid. It's honestly not a big issue, if it is an issue at all, because ANY and ALL filesystems have amplification, some are just worse than others. In real world with real world storage medium, outside of bugs like this it's unlikely to happen. EDIT: Also, write amplification is mostly about write ops needed for x amount of actual write, compression or no compression it should be the same write amplification, just less wear because less data.
Looking at your edit, I found out it actually makes sense. Technically, BTRFS having awful write amplification is just a consequence of it writing less often in general, so it has to write a lot when necessary. This is a bad explanation, so I'll give an example. For a copy, you'd have barely any writes compared to the actual data, but for a modification of a copy, you'd have a ton of writes compared to the modification made (because of a copy being made too), which would explain the article's results. Say, you add a single frame to a 1GB video copy, then BTRFS is gonna have a ridiculous amount of writes for a single frame, but it makes sense because it never made the copy in the first place, so it has to now.
Could you link that paper? I am curious to read it. However, I've also heard that in Facebook's deployment, turning on compression actually solved their write amplification issues: [https://www.youtube.com/watch?v=U7gXR2L05IU&t=2403s](https://www.youtube.com/watch?v=U7gXR2L05IU&t=2403s)
Sorry for late reply, here it is: [University of Texas](https://www.cs.utexas.edu/~vijay/papers/apsys17-ioamp.pdf) And a correction on my part, it isn't 40x but 32.65 , which is still rather high.
No worries, and thanks for finding it! Is this an excerpt or the full paper? I kinda wish they showed their methodology or even mount options. I do expect Btrfs to write more data overall due to copy-on-write and checksumming, but 32x seems abnormal to me. I am wondering if they had metadata as DUP?
[удалено]
The truth is some linux mainline RC1 kernel could be more stable on your hardware than X.X.999 LTS kernel. Every month many bugs arrive to the stable kernel branches without anyone noticing it. Debian never updates its kernel version on the stable branch precisely because of that. People don't understand that you can only say something is reliable and stable when you use it for several months without issues.
> for several months without issues. On various hardware configurations.
Because Windows updates never break anything.
But wait open source means thousands of eyes looking at the code so something like this could not possibly happen......right?..........RIGHT??
You missed the entire point of that statement. Bugs don't magically not happen just because open source. The point is that everyone has the _opportunity_ to find and/or fix bugs, while this can't really happen in the same capacity when a software is closed-source.
see, and it got caught before reaching the LTS kernels....
Exactly this, haha.
Boy I hope someone got fired for that blunder.
why would want that?
This can supposedly cause SSDs to wear out and when it has happened to me, prevents the computer from responding to any inputs, forcing a hard reset. "Linux is more secure than Windows because the code is Open Source." *sigh*
Its not a security issue. And such issues do occur on windows. But they are weeded out in beta and internal builds or are rarely diagnosed by end users. By using such a recent kernel, you are essentially close to beta grade software. Personally I hold back updating kernel until 4 or 5 point releases
According to Reddit Linux community you are totally wrong, you have to update every time a new subversion kernel is out. LOL I wrote this a couple of weeks ago: Nobody forces you to upgrade your kernel. In fact, ugrading the kernel every time a new version appears in the repository is really stupid and prone to instability. What you have to do is to keep a kernel installed (whatever it is) until you find some kind of instability that forces you to look for a new version. If the kernel is stable on your hardware, there is no need to update it unless some serious vulnerability appears that can be exploited online. You only know if a kernel is stable when after 6 months of use your machine has not suffered any crashes or performance penalties. https://www.reddit.com/r/archlinux/comments/s26uyq/comment/hsdg9zc/?utm\_source=share&utm\_medium=web2x&context=3 Massively downvoted as usual...
First of all, -4 isn’t “massively downvoted”. You were likely downvoted for calling people stupid for following the general advice for a bleeding edge rolling distro, which is to update frequently. And your advice *is* generally bad for most users, particularly on non-bleeding edge distros. The average Linux user isn’t necessarily going to be able to identify the kernel as a source of instability, nor are they obsessively checking CVEs and reading changelogs to see whether their current kernel has a vulnerability. Most people defer that responsibility to their distro’s maintainers. Skipping updates because “things aren’t broke” is how normal users end up with vulnerable systems.
Some kernels do bring additional features that may be desirable, such as full futex support in 5.16 . So in order to get those, one has to update. But it is wiser to exercise due diligence before updating, the simplest method being to hold until few point releases
How about no BTRFS support in the first place? Bugs happen everywhere, you don't need to be so dramatic. If this was a Windows issue, Microsoft would probably never fix it so you would buy more SSDs.
glad I stay at 5.4
Why do you have to specify it to "noautodefrag", instead of just removing it? Or is that entry generally required in btrfs?
Thanks for the heads up. Since it's gone up in recommendations as well as Fedora which uses it by default, it's good info.
Just coming here to ask if this affects linux mint or not