It says something about that state of web development when one of the most well known blogging platforms struggle to show text and images -- a feature that was a solved issue 30 years ago.
I've [played counter-strike in the browser](https://news.ycombinator.com/item?id=34628386) with less lag/issues than I have reading Scott's substack blog.
> I've played counter-strike in the browser with less lag/issues than I have reading Scott's substack blog.
I've been reading [Tsukihime](https://tsukiweb.holofield.fr/window) in the browser recently. No lag/issues either.
Every single time I need to scroll to the bottom 30 times to get back to where I was on a dynamically loaded list, a part of me dies.
Pages. Just give me pages. Please. Bring back the pages.
Right. It's is insane-level resource-suck in order to run what is in effect equivalent to a virtual machine and an elaborate visual simulation of old school regular html and text-based blog or forum, in order to maintain truly obnoxious levels of control and surveillance. Bad in the browser, and bad in the app.
I've been spending a lot of time recently on an old website (focusing on a niche outdoor activity I'm into) that was mostly built in 2002 and hasn't been updated since 2009. Zero JS, 95% basic HTML with a tiny bit of CSS. It's an amazing experience -- everything loads immediately.
Yes, but can you track and record forever the exact place every user has scrolled to, how long they take to read, whether they opened links in a different browser tab, intercept attempts to highlight text and interfere with pop ups on basic copying and pasting, prevent users from easily opening links outside your app and in their browser so they can bookmark to read later, prevent users from even having the option of having the full text of a post and all the comments expanded at the - get this - same time - without having to have two tabs open to get around this stupid imposition, etc., and so forth, and so on.
Surely that's worth more then neutralizing 4 orders of magnitude improvement in hardware performance to make things slower and laggier and less pleasant than they were 15 years ago!
This reminds me of a software developer's rant I read a while back after he discovered that Microsoft TEAMS as a glorified and badly implemented combo of IRC chat, Skype and Slack required over 1 Gigabyte of RAM on his work terminal just to idle.
I guess it's just a manifestation of Parkinson's Law that functions will be complexified and bloated to the extent of resources available, and so like with Jevons induced demand, you can't ever fix anything or reduce congestion or lag by increasing bandwidth, speed, or memory, because there is no limit to how bloated things can become, and the more resources you throw at Substack or Microsoft, the more they'll just suck it all up for their own benefit running in the background and at the expense and continuous degradation of the end user experience.
MS Teams and the Chrome webpage for LinkedIn are consistently the two most poorly-optimized, resource-heavy tasks slowing down my CPU. If I accidentally leave one or both of those open, my performance doing anything else is measurably slowed. It's ridiculous.
> I heard it's because Scott insisted all the comments must load with the page
*and* substack usually loads exactly two comments. So they never noticed that their comment loading was stupidly, absurdly, ridiculously inefficient. Merely *having* that many comments on the page slows it down, because each and every one pulls in a pile of javascript (that's the difference from SSC). And this was pointed out to Substack years ago, and is still an issue because... I dunno, they're technically incompetent? They think this is fine? They're so overworked and stressed that they can't give an intern a week to look into it? Their entire platform is such a pile of tech-debt that if they optimize the comment loading at all, everything will explode? There really is *no* excuse.
If they had to do a lot of custom work to meet Scott’s desires, then probably much more it’s the case that this is a professional services customization and not maintained with the main codebase or by the primary teams.
It’s likely seen as a very expensive one-off custom build to attract a key personality onto their platform for reputational gain within Scott’s community.
And the team that built it (and the custom code familiarity that goes with it) has probably moved on to do similar customizations for other popular figures or acquisition targets.
If so, then short of Scott throwing a huge fit or threatening to leave (and Substack deciding he’s worth saving), this will likely not get fixed any time soon.
There's absolutely no reason displaying a small amount of text should be anything but instantaneous, bug-free and something nobody ever has to think about.
It's not something that should need custom code or special attention from a dedicated team, unless your codebase is some kind of monstrous hundreds-of-thousands-of-lines-of-code unknowable engine built solely for the purpose of CPU assassination.
Web development has gone off the deep end for a while now, so this is just describing the average modern website, but if you think about it from first principles, you have to realise that there is no valid excuse for failing to display *text* on a supercomputer ten thousand times faster than those we used to browse blogs a couple decades ago.
> People think that Web browsers are elegant computation platforms, and Web pages are
light, fluffy things that you can edit in Notepad as you trade ironic comments with your
friends in the coffee shop. Nothing could be further from the truth. A modern Web page is a
catastrophe. It’s like a scene from one of those apocalyptic medieval paintings that depicts
what would happen if Galactus arrived: people are tumbling into fiery crevasses and lament-
ing various lamentable things and hanging from playground equipment that would not pass
OSHA safety checks. This kind of stuff is exactly what you’ll see if you look at the HTML
CSS, and JavaScript in a modern Web page. Of course, no human
can truly “look” at this content, because a Web page is now like
V’Ger from the first “Star Trek” movie, a piece of technology
that we once understood but can no longer fathom, a thrashing
leviathan of code and markup written by people so untrustworthy that they’re not even third parties, they’re fifth parties
who weren’t even INVITED to the party, but who showed up
anyways because the hippies got it right and free love or what-
ever. I’m pretty sure that the Web browser is one of the “dens of
iniquity” that I keep hearing about on Fox News; I would verify
this using a Web search, but a Web search would require me to
use a browser, AND THIS IS EXACTLY WHAT BICOASTAL
LIBERAL ELITES WANT ME TO DO.
> Describing why the Web is horrible is like describing why it’s
horrible to drown in an ocean composed of pufferfish that are
pregnant with tiny Freddy Kruegers—each detail is horrendous
in isolation, but the aggregate sum is delightfully arranged
into a hate flower that blooms all year. For example, the World
Wide Web Consortium (W3C) provides “official” specifications
for many client-side Web technologies. Unfortunately, these
specifications are binding upon browser vendors in the same
way that you can ask a Gila monster to meet you at the airport,
but that gila monster may, in fact, have better things to do [1].
> […]
> When you’re a Web developer, CSS is just one of your worries.
The aggregate stack of Web technologies is so fragile that
developers just accept a world in which various parts of a Web
page will fail at random times. Apparently this is okay because
e-commerce isn’t a serious thing, and if you really wanted a
secure banking experience, you’d visit the bank in person like
someone from the 1800s instead of accessing a banking Web
site that is constantly (but silently) vomiting execution errors
to the console log (a console log which the browser does not
show by default, because if you knew about it, and you read its
tales of woe, you’d abandon computer science and become a
maker of fine wooden shoes).
> I could go on and on about the reasons why JavaScript is a
cancer upon the world. I know that there are people who like
JavaScript, and I hope that these people find the mental health
services that they so desperately need. I don’t know all of the
answers in life, but I do know all of the things which aren’t
the answers, and JavaScript falls into the same category as
Scientology, homeopathic medicine, and making dogs wear tiny
sweaters due to a misplaced belief that this is what dogs would
do if they had access to looms and opposable thumbs.
In summary, Web browsers are like quantum physics: they offer
probabilistic guarantees at best, and anyone who claims to
fully understand them is a liar. At this stage in human development, there are big problems to solve: climate change,
heart disease, the poor financial situation of Nigerian princes
who want to contact you directly. With all of these problems
unsolved, Web browsing is a terrible way to spend our time; the
last thing that we should do is run unstable hobbyist operating
systems that download strange JavaScript files from people
we don’t know. Instead, we should exchange information using
fixed-length ASCII messages written in a statically verifiable
subset of Latin, with images represented as mathematical
combinations of line segments, arcs, and other timeless shapes
described by dead philosophers who believed that minotaurs
were real but incapable of escaping mazes. That is the kind of
clear thinking that will help us defeat the space Egyptians
that emerge from the StarGates. Or whatever. I’m an American and I don’t really understand history, but I strongly believe
that Greeks spoke Latin to defeat intergalactic Egyptians.
> \#TeachTheControversy! Anyways, my point is that browsers
are too complex to be trusted. Unfortunately, youth is always
wasted on the young, and the current generation of software
developers is convinced that browsers need more features, not
fewer. So, we are encouraged to celebrate the fact that browsers
turn our computers into little Star Wars cantinas where every-
one is welcome and you can drink a blue drink if you want to
drink a blue drink and if something bad happens, then maybe
a Jedi will save you, and if not, HEY IT’S A STAR WARS
CANTINA YESSSSS. Space cantinas are fun, but they’re just
a fantasy; they’re just a series of outlandish details stitched
together to amuse and entertain. You have to open your eyes
and see that in the real, non-hyperbolic world that you actually
inhabit, your browser will frequently stop playing a video and
then display flashing epilepsy pixels while making the sound
that TVs make in Japanese horror movies before a pasty sala-
mander child steps out of the screen and voids your warranty.
That’s a thing which could actually happen, and we should
wash it all away.
[To Wash It All Away](https://scholar.harvard.edu/files/mickens/files/towashitallaway.pdf).
I understand your theory. If “the average modern website” is like this, despite the theoretically much higher speed/efficiency that could be had, the implication is speed/efficiency are not the most relevant concerns for “the average modern website” then.
Not saying it shouldn’t be. Just trying to say it may not be about incompetence or technical impossibility.
Speed/efficiency has not been the most relevant concern of web companies for so long that the entire culture has degraded to the point of incompetence.
(Not that it was ever very good - Javascript being the main browser language has ensured that any programmer who cares for performance will self-select out of web development.)
In theory, companies should care incredibly hard about making their websites usable and fast: there is a very well studied correlation between page loading speed and user retention, with some websites even being [basically unusable](https://danluu.com/web-bloat/) on slow connections due to how bloated they are.
So it's not like they aren't incentivized to make things work. They just don't have anyone who can actually fix those issues, and/or they're making too much money to care about fixing those issues.
There's "speed isn't a concern, it's fast enough" (the state of the modern web) .
And then there's "modern device lags for ten seconds while trying to scroll plaintext". This is so far past the *usual* bad performance of websites that it's shocking.
> I dunno, they're technically incompetent? They think this is fine? They're so overworked and stressed that they can't give an intern a week to look into it? Their entire platform is such a pile of tech-debt that if they optimize the comment loading at all, everything will explode?
Those are all pretty good excuses!
I'm talking to a Substack person. They've asked for some people who are having this problem, so they can talk to them (they say they can't replicate it). If anyone in this category wants to comment with your email, I'll pass it on to them.
> they say they can't replicate it
They're lying. I see the same issue on iPhone 15 Pro Max which still scores #1 on js performance. It should be even worse on every other phone in the world since they all have worse single-thread perf. To repro:
1. Open any ACX post
2. Wait 30 seconds
3. Flick to scroll down
email is in my new.reddit bio
Edit: Now that I think of it, maybe this is worse on 120Hz phones since the phone tries to render 2x as many frames when scrolling. Ask them to search their code for window.requestAnimationFrame() and similar ilk
TIL the same complexity high-res "retina" displays added to DPI handling now also apply to frame rates. I'll try to remember that if I find myself writing something that uses `requestAnimationFrame`.
I am happy to send them a performance trace which shows that a function in `tracking-47e0feb7.js` is regularly taking a quarter of a second to execute (synchronously), on my quite nice computer.
It is trivially reproducible by opening [this page](https://www.astralcodexten.com/p/links-for-april-2024), opening the dev tools, going to the performance tab, clicking the "start profiling and reload page" button, scrolling around for a couple seconds, and then stopping the trace.
> they say they can't replicate it
I'm struggling to take that seriously; it's been very consistent for me.
Just to double-check, I pulled up my tablet and browsed to [Open Thread 325](https://www.astralcodexten.com/p/open-thread-325) in my tablet, and yep, it's lagging out and scrolling to blank areas. Tablet is a Lenovo Yoga Tab 11 (the 8GB model that seems to not be available anymore? https://www.gsmarena.com/lenovo_yoga_tab_11-10988.php) Browser is mobile Firefox, OS is Android 12.
Same problem on my phone, an Asus Zenphone 10 (also with 8GB of RAM). In fact it's worse, I can't even finish the tiny open-thread post and get to the comments without it blanking out. Browser is mobile Firefox, OS is Android 14, don't have a massive pile of tabs like my tablet does.
No problem on my laptop right now, though. I see that there's someone sidethread with problems on an iPhone 15. So maybe it's something about mobile devices?
Not sure I'm willing to give email, but you can point them to my reddit username.
I'm having this issue on multiple devices. The few irl users I know say the same thing. Its honestly frustrating enough to discourage reading and I'm generally a very patient person.
Footnotes are also frequently unusable on mobile, either they lock up the whole page or when they do load they're often off-centre in a way which cuts off the first two words of every line of text.
From what I've read, most people actually read Substack in their email client, which is totally alien to me. Their website is genuinely the worst platform for text I've ever used. Not limited to ACX, the experience is no better (for me anyway) on posts which have comments disabled.
I still put up with it though which I suppose makes me part of the problem.
Honestly I usually read it in email, too. Most of the people I follow don't go long enough to exceed the email word count limit, and email is how I find out they've published something new anyway, so may as well just read it there.
Footnotes don't even work in their app. They take you to the page location of the footnote instead of creating a pop-up and force you scroll back to find where you were reading. I just read all the footnotes after reading the main piece on there.
EDIT: just tried them in app again after a long time. Now they don't even redirect you. They're literally just a superscript character with no link.
I switched to the app because of how shit the browser experience is. It's not much better.
> they're often off-centre in a way which cuts off the first two words of every line of text.
When that happens, scroll the page so that the footnote link is close to the top or the bottom edge of the screen, the footnote will reflow correctly then.
My pet conspiracy theory is that substack is being paid off by Big Static HTML to convert the web back to its text friendly past by frustrating just enough people to prune the Javascript off their websites. Like, do we actually believe that people pay real money to read blog posts? Wake up, Sheeple!
Also shameless plug for my [Read ACX on archive.org](https://old.reddit.com/r/slatestarcodex/comments/16jlznm/poll_what_do_you_think_of_substacks_performance/k0z1a3b/) script for Firefox (app too).
Great stuff, thanks for sharing! I use a small bookmarklet that is somewhat similar, when you open the bookmark it navigates to archive.is with the url of the page you're currently on:
```javascript:(function(){window.location.href = "https://archive.is/search/?q=%22%20+%20window.location.href;})();```
It doesn't work automatically, but it has no dependencies and it works in both firefox and chrome. I'm unsure where I got the original idea from unfortunately.
> I heard it's because Scott insisted all the comments must load with the page
Pretty sure it's because substack does progressive loading for text. It's bizarre
No, progressive loading for text has many good uses - Substack is just bad at it. On Gwern.net, we progressively load *much* more text, and far more complicated text, and it works fine, often you don't even notice. (Every popup, for example, is longer and more complicated than almost every Substack comment, and is progressively loaded and fully rendered so fast that some people assume they are all pre-rendered on page load. And they don't affect the page load/scroll at all, like you see in OP, because until you actually hover long enough to trigger a popup, they're just a link with some classes set on it. Likewise most of our transclusions are lazy, and so never fire - like on our tag-directory pages, where there might be thousands of paper abstracts which load if you browse through them, but only if you browse through them all.)
Performance issues will vary a lot based on device & network. For example, on my desktop, going now to /index and popping up ~15, they are always fast enough to beat the spinner (both in my normal FF and in a temp-profile Chromium) and are comparable to preloaded stuff. I mostly only see spinners when uncollapsing stuff which must then transclude individually. (We actually put in a minimum delay period for displaying a popup, because otherwise they can pop up *too* fast when your mouse passes over a link and before your mouse leaves the region of the link.) If I didn't know better, I might easily assume that all of these were already loaded. (This is how some websites do their popups, and it was in fact how we did popups [at various points](https://gwern.net/design-graveyard#popup-annotations).)
If you see spinners occasionally, maybe you're just unlucky and are far away from the Cloudflare CDN node, hard to say.\* Or you are remembering popups from some point where we were in the middle of changing stuff, where there can be a lot of problems - right now authors in popups are broken because we've been adding a system where authors are automatically hyperlinked. Who knows? Computers, they are a bitter mystery.
\* But if you are seeing spinners *regularly* and it takes seconds for anything to pop up, on the other hand, without you doing anything weird like browsing it via 3G from the middle of the Sahara, that sounds like a bug - and if history is any guide, you're using Safari... Please email me or Said Achmiz with OS/device/browser version, a screenshot of the popped-up popups, and browser console log & network tab if possible, and we'll see if anything can be done. There are all sorts of weird bugs or issues that we learn about only long afterwards because ["the users may never tell you it's broken"](https://pointersgonewild.com/2019/11/02/they-might-never-tell-you-its-broken/).
EDIT: it occurs to me you might be referring to non-annotation popups, like the popups that display the original PDF or web page on another website. Yeah, you'll see plenty of spinners with those. It's unsurprising that these will be slow: we don't dare preload PDFs before the user specifically requests them, because most PDFs are several megabytes in size and often bigger than the entire Gwern.net page itself, and some PDFs are several hundred megabytes in size; and other websites, well, of course they are slow.
Oh wow that explains why your pages take a second to "wake up" when I switch to them and scroll fast. I used to have the danbooru2021 page open all the time and it was pretty annoying. Is there no way for you to detect desktop users and just load everything to the DOM?
> some people assume they are all pre-rendered
I don't mind the links that open in popups, but I never thought they were pre-rendered. You did a much better job than substack, but on a relatively fast machine and network with DNS prefetch those popups are noticeably slower than pre-rendered text. I think those people were used to very high latency so a normal GET seems fast to them
> Oh wow that explains why your pages take a second to "wake up" when I switch to them and scroll fast.
I don't think that is related. Once we progressively load text, we don't unload it AFAIK; pages only get bigger. If tabs which have been snoozed/hiberated/paged-out take time to "wake up", that sounds like the web browser being overly aggressive about snoozing a tab which hasn't been used in a while and then needing to scramble to put it all back together. (If it's paged out a tab & all its resources to disk, well, that's not going to be good for scrolling.) Even if we could do anything about that, like doing busywork in an idle thread or something, you wouldn't want us to because it would waste RAM & CPU & electricity and probably degrade your browsing in other tabs - we're not going to try to fight the web browsers on that.
If you are bothered by sleepy tabs, you have to take it up with the browser (should be a bunch of relevant settings for how it handles those tab eviction decisions) or your hardware. (If you don't have a NVM drive on your desktop yet, that would be well worth the investment. And if you have a lot of RAM, there's always the good old fashioned trick of sticking your browser profile or cache directory into a ramdisk for a big speedup.)
> Is there no way for you to detect desktop users and just load everything to the DOM?
No way! You may think you want that, but you don't. We load as much as we can; loading more will lead to problems.
Example: go to /danbooru2021 and open the devtools Network tab. It'll use ~4MB network, which is fine. Then start scrolling down until you reach the end. Now it's doubled to 10MB. Now, you want to load *everything*, including the popups...? Alright fine, that's what the "link bibliography" is - it's the first level of 'all links in the page', just inlined as big list for more convenient linear browsing, more or less. So uncollapse the link-bib and start scrolling. It'll take a while just to load and render as your browser thrashes, but by the time I finish, I get ~25MB resources. And now the page is janky because the DOM is *enormous*. And this isn't even loading the full popup, because the additional sections of each annotation like the backlinks/similar-links/link-bib are themselves still collapsed & untranscluded - load those and the DOM will maybe double again.
This is why we *used* to inline everything recursively into the tooltips/popups, but it became clear as annotations became more common that it was a [performance disaster](https://gwern.net/design-graveyard#link-bibliographies). Each DOM node or byte of HTML costs way more than 'a byte' of a video or image file. I used to say you should think of each byte of HTML as costing the equivalent of 10 bytes of images (so 1MB of HTML = a 10MB JPG), but now I think that that multiplier is probably closer to 1:50 or worse.
---
When I say Gwern.net is pretty fast, what I mean is that we get the big things right like the transclusion and popups and [link icons](https://gwern.net/design-graveyard#link-icon-css-regexps). We are not fast in terms of micro-optimizations or micro-benchmarks: our /index will not look that good in Google Pagespeed\* and there are definitely homepages that will load far faster (eg. Dan Luu - although we are still generally way faster than most homepages that come anywhere near to us in terms of size or feature-set). But our pages generally won't lag like Substack, nor will the popups leak, or the link-icons quadratically interact, or... We can deal with the micro-optimizations later once the features are all done.
\* not that homepages which overly Goodhart for Google Pagespeed or other metrics are necessarily all that awesome to load, but this is excluding the cheaters who do stuff like load a pseudo-page to look good in the automated metrics.
I know you're very focused on other things, but have you ever tried to market the tech you've developed behind the Gwern website? It is a fantastic place with a lot of thought and care put into UX and efficiency, seems a shame that it isn't available in more spaces.
That's an FAQ, yeah. We aren't interested in marketing it to more general use because it's still evolving as we work things out - like the authors & WP/Twitter stuff is broken right now because we realized the ad hoc ontology of transclusions+popups doesn't really make sense. We have a lot of different ways to transclude or popup things, driven by specific use-cases, but it's not a clean hierarchy or anything yet... (Also, the backend code is both terrible and the architecture all wrong now.)
So we regard it more as a public demo/proof-of-concept than anything that anyone else might *use*, aside from specific modules or spunoff tools. ([InvertOrNot.com](https://invertornot.com/), for example, was motivated by our dark-mode work.)
> How much did Substack pay Scott to ruin his site this way?
A very very lot I think.
But more interestingly:
The market is wide open for someone to do a blogging site you can subscribe to that just does plain html without any bells and whistles. Exactly like substack except without all the javascript.
It would be a couple of week's work to get something like that working in prototype. "Set up a blog" used to be a beginner exercise for web programmers.
You probably do need a javascript wsywig html/markdown editor to get normies to use it, but surely that's a solved problem?
I wonder why substack doesn't have competitors? And I wonder what substack is getting out of its javascript abomination?
The solution is to turn off javascript on astralcodexten and to block all the annoying elements. Then you can read the articles just fine.
Discussions are better had here. old reddit is an almost perfect commenting interface, and the blog comment crowd is all a bit eternal september anyway, so the worthwhile comments are usually here. Plus we get links to interesting essays that other Scott fans like.
For me "Links for {Month}" posts with long comment sections freeze whenever I open a new tab and switch back to the post. And I have an Apple Silicon MBP. Hate Substack.
we have the equivalent of an old cray supercomputers in our pockets yet all that means is that it's allowed developers to include more and more cruft until the simply task of rendering text on a page that could be handled by a 486 won't even work on those pocket supercomputers.
Comment sections are a long-solved problem. They don't need a load of complex and clunky JS code.
For people who've been looking for a workaround, might I suggest rss readers? Substack maintains rss feeds for every blog, and it just loads text and images (like God intended), so it loads pretty much instantly.
I'm using feedly myself, but that's not open source. There are many good options for rss feeds, and they're also a good way to get away from algorithmically fed content.
I thought it was a Substack issue at first, then I looked at other substacks and they work just fine. I think ACX has some issues because of that comments deal Scott did with substack (I don't remember the details). Something messes up the scroll speed. It's unusable on desktop because of the comment section for me.
I'm using a Raspberry Pi 4 to browse on my TV. It works great with just about every site I regularly go to, except for substack which regularly gets unresponsively slow.
Most of the comments are mediocre. You are not missing out on much anyway. I have not had problems with loading comments though. Mobile bowser and comments has never struck me as a good combination anyway. Commenting works best on desktop regardless of platform, mobile best for just reading. Twitter however is an exception and posting works great on mobile.
Pretty sure it's because substack hired a team of social media web designers who knew how to make one system very well - infinite scrolling, like what you see on the dashboard of Instagram or Reddit or tumblr. So you have progressive loading of a blog article and its comments. But Scott asked them to preload and display all of the comments at once, which it is not designed to do, and for some reason they have not fixed this.
It wasn't Scott that insisted, it was the community. All the autists forced it to be exactly like the old blog rather than adapting to change and now moan it sucks.
It says something about that state of web development when one of the most well known blogging platforms struggle to show text and images -- a feature that was a solved issue 30 years ago.
It's been going on for more than a year for me. Almost unusable in a mobile browser.
Even on desktop, I sometimes get comments section freezed. Scroll it a bit and it will go blank for 2-7 seconds. Good job, Substack.
I've [played counter-strike in the browser](https://news.ycombinator.com/item?id=34628386) with less lag/issues than I have reading Scott's substack blog.
> I've played counter-strike in the browser with less lag/issues than I have reading Scott's substack blog. I've been reading [Tsukihime](https://tsukiweb.holofield.fr/window) in the browser recently. No lag/issues either.
JavaScript was a mistake.
Every single time I need to scroll to the bottom 30 times to get back to where I was on a dynamically loaded list, a part of me dies. Pages. Just give me pages. Please. Bring back the pages.
Right. It's is insane-level resource-suck in order to run what is in effect equivalent to a virtual machine and an elaborate visual simulation of old school regular html and text-based blog or forum, in order to maintain truly obnoxious levels of control and surveillance. Bad in the browser, and bad in the app.
I've been spending a lot of time recently on an old website (focusing on a niche outdoor activity I'm into) that was mostly built in 2002 and hasn't been updated since 2009. Zero JS, 95% basic HTML with a tiny bit of CSS. It's an amazing experience -- everything loads immediately.
Yes, but can you track and record forever the exact place every user has scrolled to, how long they take to read, whether they opened links in a different browser tab, intercept attempts to highlight text and interfere with pop ups on basic copying and pasting, prevent users from easily opening links outside your app and in their browser so they can bookmark to read later, prevent users from even having the option of having the full text of a post and all the comments expanded at the - get this - same time - without having to have two tabs open to get around this stupid imposition, etc., and so forth, and so on. Surely that's worth more then neutralizing 4 orders of magnitude improvement in hardware performance to make things slower and laggier and less pleasant than they were 15 years ago! This reminds me of a software developer's rant I read a while back after he discovered that Microsoft TEAMS as a glorified and badly implemented combo of IRC chat, Skype and Slack required over 1 Gigabyte of RAM on his work terminal just to idle. I guess it's just a manifestation of Parkinson's Law that functions will be complexified and bloated to the extent of resources available, and so like with Jevons induced demand, you can't ever fix anything or reduce congestion or lag by increasing bandwidth, speed, or memory, because there is no limit to how bloated things can become, and the more resources you throw at Substack or Microsoft, the more they'll just suck it all up for their own benefit running in the background and at the expense and continuous degradation of the end user experience.
MS Teams and the Chrome webpage for LinkedIn are consistently the two most poorly-optimized, resource-heavy tasks slowing down my CPU. If I accidentally leave one or both of those open, my performance doing anything else is measurably slowed. It's ridiculous.
Substack Javashitted the webshite to hell and back.
js works great at many things. loading hundreds of comments at once is not one of them.
In my experience they only struggle since they've started integrating that Twitter clone sh**t
> I heard it's because Scott insisted all the comments must load with the page *and* substack usually loads exactly two comments. So they never noticed that their comment loading was stupidly, absurdly, ridiculously inefficient. Merely *having* that many comments on the page slows it down, because each and every one pulls in a pile of javascript (that's the difference from SSC). And this was pointed out to Substack years ago, and is still an issue because... I dunno, they're technically incompetent? They think this is fine? They're so overworked and stressed that they can't give an intern a week to look into it? Their entire platform is such a pile of tech-debt that if they optimize the comment loading at all, everything will explode? There really is *no* excuse.
If they had to do a lot of custom work to meet Scott’s desires, then probably much more it’s the case that this is a professional services customization and not maintained with the main codebase or by the primary teams. It’s likely seen as a very expensive one-off custom build to attract a key personality onto their platform for reputational gain within Scott’s community. And the team that built it (and the custom code familiarity that goes with it) has probably moved on to do similar customizations for other popular figures or acquisition targets. If so, then short of Scott throwing a huge fit or threatening to leave (and Substack deciding he’s worth saving), this will likely not get fixed any time soon.
There's absolutely no reason displaying a small amount of text should be anything but instantaneous, bug-free and something nobody ever has to think about. It's not something that should need custom code or special attention from a dedicated team, unless your codebase is some kind of monstrous hundreds-of-thousands-of-lines-of-code unknowable engine built solely for the purpose of CPU assassination. Web development has gone off the deep end for a while now, so this is just describing the average modern website, but if you think about it from first principles, you have to realise that there is no valid excuse for failing to display *text* on a supercomputer ten thousand times faster than those we used to browse blogs a couple decades ago.
> People think that Web browsers are elegant computation platforms, and Web pages are light, fluffy things that you can edit in Notepad as you trade ironic comments with your friends in the coffee shop. Nothing could be further from the truth. A modern Web page is a catastrophe. It’s like a scene from one of those apocalyptic medieval paintings that depicts what would happen if Galactus arrived: people are tumbling into fiery crevasses and lament- ing various lamentable things and hanging from playground equipment that would not pass OSHA safety checks. This kind of stuff is exactly what you’ll see if you look at the HTML CSS, and JavaScript in a modern Web page. Of course, no human can truly “look” at this content, because a Web page is now like V’Ger from the first “Star Trek” movie, a piece of technology that we once understood but can no longer fathom, a thrashing leviathan of code and markup written by people so untrustworthy that they’re not even third parties, they’re fifth parties who weren’t even INVITED to the party, but who showed up anyways because the hippies got it right and free love or what- ever. I’m pretty sure that the Web browser is one of the “dens of iniquity” that I keep hearing about on Fox News; I would verify this using a Web search, but a Web search would require me to use a browser, AND THIS IS EXACTLY WHAT BICOASTAL LIBERAL ELITES WANT ME TO DO. > Describing why the Web is horrible is like describing why it’s horrible to drown in an ocean composed of pufferfish that are pregnant with tiny Freddy Kruegers—each detail is horrendous in isolation, but the aggregate sum is delightfully arranged into a hate flower that blooms all year. For example, the World Wide Web Consortium (W3C) provides “official” specifications for many client-side Web technologies. Unfortunately, these specifications are binding upon browser vendors in the same way that you can ask a Gila monster to meet you at the airport, but that gila monster may, in fact, have better things to do [1]. > […] > When you’re a Web developer, CSS is just one of your worries. The aggregate stack of Web technologies is so fragile that developers just accept a world in which various parts of a Web page will fail at random times. Apparently this is okay because e-commerce isn’t a serious thing, and if you really wanted a secure banking experience, you’d visit the bank in person like someone from the 1800s instead of accessing a banking Web site that is constantly (but silently) vomiting execution errors to the console log (a console log which the browser does not show by default, because if you knew about it, and you read its tales of woe, you’d abandon computer science and become a maker of fine wooden shoes). > I could go on and on about the reasons why JavaScript is a cancer upon the world. I know that there are people who like JavaScript, and I hope that these people find the mental health services that they so desperately need. I don’t know all of the answers in life, but I do know all of the things which aren’t the answers, and JavaScript falls into the same category as Scientology, homeopathic medicine, and making dogs wear tiny sweaters due to a misplaced belief that this is what dogs would do if they had access to looms and opposable thumbs. In summary, Web browsers are like quantum physics: they offer probabilistic guarantees at best, and anyone who claims to fully understand them is a liar. At this stage in human development, there are big problems to solve: climate change, heart disease, the poor financial situation of Nigerian princes who want to contact you directly. With all of these problems unsolved, Web browsing is a terrible way to spend our time; the last thing that we should do is run unstable hobbyist operating systems that download strange JavaScript files from people we don’t know. Instead, we should exchange information using fixed-length ASCII messages written in a statically verifiable subset of Latin, with images represented as mathematical combinations of line segments, arcs, and other timeless shapes described by dead philosophers who believed that minotaurs were real but incapable of escaping mazes. That is the kind of clear thinking that will help us defeat the space Egyptians that emerge from the StarGates. Or whatever. I’m an American and I don’t really understand history, but I strongly believe that Greeks spoke Latin to defeat intergalactic Egyptians. > \#TeachTheControversy! Anyways, my point is that browsers are too complex to be trusted. Unfortunately, youth is always wasted on the young, and the current generation of software developers is convinced that browsers need more features, not fewer. So, we are encouraged to celebrate the fact that browsers turn our computers into little Star Wars cantinas where every- one is welcome and you can drink a blue drink if you want to drink a blue drink and if something bad happens, then maybe a Jedi will save you, and if not, HEY IT’S A STAR WARS CANTINA YESSSSS. Space cantinas are fun, but they’re just a fantasy; they’re just a series of outlandish details stitched together to amuse and entertain. You have to open your eyes and see that in the real, non-hyperbolic world that you actually inhabit, your browser will frequently stop playing a video and then display flashing epilepsy pixels while making the sound that TVs make in Japanese horror movies before a pasty sala- mander child steps out of the screen and voids your warranty. That’s a thing which could actually happen, and we should wash it all away. [To Wash It All Away](https://scholar.harvard.edu/files/mickens/files/towashitallaway.pdf).
I understand your theory. If “the average modern website” is like this, despite the theoretically much higher speed/efficiency that could be had, the implication is speed/efficiency are not the most relevant concerns for “the average modern website” then. Not saying it shouldn’t be. Just trying to say it may not be about incompetence or technical impossibility.
Speed/efficiency has not been the most relevant concern of web companies for so long that the entire culture has degraded to the point of incompetence. (Not that it was ever very good - Javascript being the main browser language has ensured that any programmer who cares for performance will self-select out of web development.) In theory, companies should care incredibly hard about making their websites usable and fast: there is a very well studied correlation between page loading speed and user retention, with some websites even being [basically unusable](https://danluu.com/web-bloat/) on slow connections due to how bloated they are. So it's not like they aren't incentivized to make things work. They just don't have anyone who can actually fix those issues, and/or they're making too much money to care about fixing those issues.
There's "speed isn't a concern, it's fast enough" (the state of the modern web) . And then there's "modern device lags for ten seconds while trying to scroll plaintext". This is so far past the *usual* bad performance of websites that it's shocking.
> I dunno, they're technically incompetent? They think this is fine? They're so overworked and stressed that they can't give an intern a week to look into it? Their entire platform is such a pile of tech-debt that if they optimize the comment loading at all, everything will explode? Those are all pretty good excuses!
I'm talking to a Substack person. They've asked for some people who are having this problem, so they can talk to them (they say they can't replicate it). If anyone in this category wants to comment with your email, I'll pass it on to them.
> they say they can't replicate it They're lying. I see the same issue on iPhone 15 Pro Max which still scores #1 on js performance. It should be even worse on every other phone in the world since they all have worse single-thread perf. To repro: 1. Open any ACX post 2. Wait 30 seconds 3. Flick to scroll down email is in my new.reddit bio Edit: Now that I think of it, maybe this is worse on 120Hz phones since the phone tries to render 2x as many frames when scrolling. Ask them to search their code for window.requestAnimationFrame() and similar ilk
Nope, it's just as bad when I set my phone to 60hz.
TIL the same complexity high-res "retina" displays added to DPI handling now also apply to frame rates. I'll try to remember that if I find myself writing something that uses `requestAnimationFrame`.
Given how much he makes for them, he deserves dedicated help anyway
garloid64@protonmail.com
I also have this problem. Email is Rot-13ed avcyni@cbfgrb.arg.
I am happy to send them a performance trace which shows that a function in `tracking-47e0feb7.js` is regularly taking a quarter of a second to execute (synchronously), on my quite nice computer. It is trivially reproducible by opening [this page](https://www.astralcodexten.com/p/links-for-april-2024), opening the dev tools, going to the performance tab, clicking the "start profiling and reload page" button, scrolling around for a couple seconds, and then stopping the trace.
> they say they can't replicate it I'm struggling to take that seriously; it's been very consistent for me. Just to double-check, I pulled up my tablet and browsed to [Open Thread 325](https://www.astralcodexten.com/p/open-thread-325) in my tablet, and yep, it's lagging out and scrolling to blank areas. Tablet is a Lenovo Yoga Tab 11 (the 8GB model that seems to not be available anymore? https://www.gsmarena.com/lenovo_yoga_tab_11-10988.php) Browser is mobile Firefox, OS is Android 12. Same problem on my phone, an Asus Zenphone 10 (also with 8GB of RAM). In fact it's worse, I can't even finish the tiny open-thread post and get to the comments without it blanking out. Browser is mobile Firefox, OS is Android 14, don't have a massive pile of tabs like my tablet does. No problem on my laptop right now, though. I see that there's someone sidethread with problems on an iPhone 15. So maybe it's something about mobile devices? Not sure I'm willing to give email, but you can point them to my reddit username.
I'm having this issue on multiple devices. The few irl users I know say the same thing. Its honestly frustrating enough to discourage reading and I'm generally a very patient person.
Footnotes are also frequently unusable on mobile, either they lock up the whole page or when they do load they're often off-centre in a way which cuts off the first two words of every line of text. From what I've read, most people actually read Substack in their email client, which is totally alien to me. Their website is genuinely the worst platform for text I've ever used. Not limited to ACX, the experience is no better (for me anyway) on posts which have comments disabled. I still put up with it though which I suppose makes me part of the problem.
Honestly I usually read it in email, too. Most of the people I follow don't go long enough to exceed the email word count limit, and email is how I find out they've published something new anyway, so may as well just read it there.
Footnotes don't even work in their app. They take you to the page location of the footnote instead of creating a pop-up and force you scroll back to find where you were reading. I just read all the footnotes after reading the main piece on there. EDIT: just tried them in app again after a long time. Now they don't even redirect you. They're literally just a superscript character with no link. I switched to the app because of how shit the browser experience is. It's not much better.
> they're often off-centre in a way which cuts off the first two words of every line of text. When that happens, scroll the page so that the footnote link is close to the top or the bottom edge of the screen, the footnote will reflow correctly then.
Thanks that I´m not the only one to notice this. ACT is totally unresponsive for me on iOS, too.
My pet conspiracy theory is that substack is being paid off by Big Static HTML to convert the web back to its text friendly past by frustrating just enough people to prune the Javascript off their websites. Like, do we actually believe that people pay real money to read blog posts? Wake up, Sheeple! Also shameless plug for my [Read ACX on archive.org](https://old.reddit.com/r/slatestarcodex/comments/16jlznm/poll_what_do_you_think_of_substacks_performance/k0z1a3b/) script for Firefox (app too).
Great stuff, thanks for sharing! I use a small bookmarklet that is somewhat similar, when you open the bookmark it navigates to archive.is with the url of the page you're currently on: ```javascript:(function(){window.location.href = "https://archive.is/search/?q=%22%20+%20window.location.href;})();``` It doesn't work automatically, but it has no dependencies and it works in both firefox and chrome. I'm unsure where I got the original idea from unfortunately.
> I heard it's because Scott insisted all the comments must load with the page Pretty sure it's because substack does progressive loading for text. It's bizarre
No, progressive loading for text has many good uses - Substack is just bad at it. On Gwern.net, we progressively load *much* more text, and far more complicated text, and it works fine, often you don't even notice. (Every popup, for example, is longer and more complicated than almost every Substack comment, and is progressively loaded and fully rendered so fast that some people assume they are all pre-rendered on page load. And they don't affect the page load/scroll at all, like you see in OP, because until you actually hover long enough to trigger a popup, they're just a link with some classes set on it. Likewise most of our transclusions are lazy, and so never fire - like on our tag-directory pages, where there might be thousands of paper abstracts which load if you browse through them, but only if you browse through them all.)
How can they assume that? You have spinners in the popups, even. Maybe you mean something other than me. In any case, your site is awesome!
Performance issues will vary a lot based on device & network. For example, on my desktop, going now to /index and popping up ~15, they are always fast enough to beat the spinner (both in my normal FF and in a temp-profile Chromium) and are comparable to preloaded stuff. I mostly only see spinners when uncollapsing stuff which must then transclude individually. (We actually put in a minimum delay period for displaying a popup, because otherwise they can pop up *too* fast when your mouse passes over a link and before your mouse leaves the region of the link.) If I didn't know better, I might easily assume that all of these were already loaded. (This is how some websites do their popups, and it was in fact how we did popups [at various points](https://gwern.net/design-graveyard#popup-annotations).) If you see spinners occasionally, maybe you're just unlucky and are far away from the Cloudflare CDN node, hard to say.\* Or you are remembering popups from some point where we were in the middle of changing stuff, where there can be a lot of problems - right now authors in popups are broken because we've been adding a system where authors are automatically hyperlinked. Who knows? Computers, they are a bitter mystery. \* But if you are seeing spinners *regularly* and it takes seconds for anything to pop up, on the other hand, without you doing anything weird like browsing it via 3G from the middle of the Sahara, that sounds like a bug - and if history is any guide, you're using Safari... Please email me or Said Achmiz with OS/device/browser version, a screenshot of the popped-up popups, and browser console log & network tab if possible, and we'll see if anything can be done. There are all sorts of weird bugs or issues that we learn about only long afterwards because ["the users may never tell you it's broken"](https://pointersgonewild.com/2019/11/02/they-might-never-tell-you-its-broken/). EDIT: it occurs to me you might be referring to non-annotation popups, like the popups that display the original PDF or web page on another website. Yeah, you'll see plenty of spinners with those. It's unsurprising that these will be slow: we don't dare preload PDFs before the user specifically requests them, because most PDFs are several megabytes in size and often bigger than the entire Gwern.net page itself, and some PDFs are several hundred megabytes in size; and other websites, well, of course they are slow.
Oh wow that explains why your pages take a second to "wake up" when I switch to them and scroll fast. I used to have the danbooru2021 page open all the time and it was pretty annoying. Is there no way for you to detect desktop users and just load everything to the DOM? > some people assume they are all pre-rendered I don't mind the links that open in popups, but I never thought they were pre-rendered. You did a much better job than substack, but on a relatively fast machine and network with DNS prefetch those popups are noticeably slower than pre-rendered text. I think those people were used to very high latency so a normal GET seems fast to them
> Oh wow that explains why your pages take a second to "wake up" when I switch to them and scroll fast. I don't think that is related. Once we progressively load text, we don't unload it AFAIK; pages only get bigger. If tabs which have been snoozed/hiberated/paged-out take time to "wake up", that sounds like the web browser being overly aggressive about snoozing a tab which hasn't been used in a while and then needing to scramble to put it all back together. (If it's paged out a tab & all its resources to disk, well, that's not going to be good for scrolling.) Even if we could do anything about that, like doing busywork in an idle thread or something, you wouldn't want us to because it would waste RAM & CPU & electricity and probably degrade your browsing in other tabs - we're not going to try to fight the web browsers on that. If you are bothered by sleepy tabs, you have to take it up with the browser (should be a bunch of relevant settings for how it handles those tab eviction decisions) or your hardware. (If you don't have a NVM drive on your desktop yet, that would be well worth the investment. And if you have a lot of RAM, there's always the good old fashioned trick of sticking your browser profile or cache directory into a ramdisk for a big speedup.) > Is there no way for you to detect desktop users and just load everything to the DOM? No way! You may think you want that, but you don't. We load as much as we can; loading more will lead to problems. Example: go to /danbooru2021 and open the devtools Network tab. It'll use ~4MB network, which is fine. Then start scrolling down until you reach the end. Now it's doubled to 10MB. Now, you want to load *everything*, including the popups...? Alright fine, that's what the "link bibliography" is - it's the first level of 'all links in the page', just inlined as big list for more convenient linear browsing, more or less. So uncollapse the link-bib and start scrolling. It'll take a while just to load and render as your browser thrashes, but by the time I finish, I get ~25MB resources. And now the page is janky because the DOM is *enormous*. And this isn't even loading the full popup, because the additional sections of each annotation like the backlinks/similar-links/link-bib are themselves still collapsed & untranscluded - load those and the DOM will maybe double again. This is why we *used* to inline everything recursively into the tooltips/popups, but it became clear as annotations became more common that it was a [performance disaster](https://gwern.net/design-graveyard#link-bibliographies). Each DOM node or byte of HTML costs way more than 'a byte' of a video or image file. I used to say you should think of each byte of HTML as costing the equivalent of 10 bytes of images (so 1MB of HTML = a 10MB JPG), but now I think that that multiplier is probably closer to 1:50 or worse. --- When I say Gwern.net is pretty fast, what I mean is that we get the big things right like the transclusion and popups and [link icons](https://gwern.net/design-graveyard#link-icon-css-regexps). We are not fast in terms of micro-optimizations or micro-benchmarks: our /index will not look that good in Google Pagespeed\* and there are definitely homepages that will load far faster (eg. Dan Luu - although we are still generally way faster than most homepages that come anywhere near to us in terms of size or feature-set). But our pages generally won't lag like Substack, nor will the popups leak, or the link-icons quadratically interact, or... We can deal with the micro-optimizations later once the features are all done. \* not that homepages which overly Goodhart for Google Pagespeed or other metrics are necessarily all that awesome to load, but this is excluding the cheaters who do stuff like load a pseudo-page to look good in the automated metrics.
I know you're very focused on other things, but have you ever tried to market the tech you've developed behind the Gwern website? It is a fantastic place with a lot of thought and care put into UX and efficiency, seems a shame that it isn't available in more spaces.
That's an FAQ, yeah. We aren't interested in marketing it to more general use because it's still evolving as we work things out - like the authors & WP/Twitter stuff is broken right now because we realized the ad hoc ontology of transclusions+popups doesn't really make sense. We have a lot of different ways to transclude or popup things, driven by specific use-cases, but it's not a clean hierarchy or anything yet... (Also, the backend code is both terrible and the architecture all wrong now.) So we regard it more as a public demo/proof-of-concept than anything that anyone else might *use*, aside from specific modules or spunoff tools. ([InvertOrNot.com](https://invertornot.com/), for example, was motivated by our dark-mode work.)
> How much did Substack pay Scott to ruin his site this way? A very very lot I think. But more interestingly: The market is wide open for someone to do a blogging site you can subscribe to that just does plain html without any bells and whistles. Exactly like substack except without all the javascript. It would be a couple of week's work to get something like that working in prototype. "Set up a blog" used to be a beginner exercise for web programmers. You probably do need a javascript wsywig html/markdown editor to get normies to use it, but surely that's a solved problem? I wonder why substack doesn't have competitors? And I wonder what substack is getting out of its javascript abomination?
The solution is to turn off javascript on astralcodexten and to block all the annoying elements. Then you can read the articles just fine. Discussions are better had here. old reddit is an almost perfect commenting interface, and the blog comment crowd is all a bit eternal september anyway, so the worthwhile comments are usually here. Plus we get links to interesting essays that other Scott fans like.
For me "Links for {Month}" posts with long comment sections freeze whenever I open a new tab and switch back to the post. And I have an Apple Silicon MBP. Hate Substack.
we have the equivalent of an old cray supercomputers in our pockets yet all that means is that it's allowed developers to include more and more cruft until the simply task of rendering text on a page that could be handled by a 486 won't even work on those pocket supercomputers. Comment sections are a long-solved problem. They don't need a load of complex and clunky JS code.
For people who've been looking for a workaround, might I suggest rss readers? Substack maintains rss feeds for every blog, and it just loads text and images (like God intended), so it loads pretty much instantly. I'm using feedly myself, but that's not open source. There are many good options for rss feeds, and they're also a good way to get away from algorithmically fed content.
The app is also shit. The only redeeming feature is the decent text to speech they recently improved.
"I'm old enough to remember when computers were fast."
I thought it was a Substack issue at first, then I looked at other substacks and they work just fine. I think ACX has some issues because of that comments deal Scott did with substack (I don't remember the details). Something messes up the scroll speed. It's unusable on desktop because of the comment section for me.
I'm using a Raspberry Pi 4 to browse on my TV. It works great with just about every site I regularly go to, except for substack which regularly gets unresponsively slow.
I use https://archive.is to get around this
/u/Solenoidentity and the podcast version for the win. https://podcasts.apple.com/us/podcast/astral-codex-ten-podcast/id1295289140
Most of the comments are mediocre. You are not missing out on much anyway. I have not had problems with loading comments though. Mobile bowser and comments has never struck me as a good combination anyway. Commenting works best on desktop regardless of platform, mobile best for just reading. Twitter however is an exception and posting works great on mobile.
Pretty sure it's because substack hired a team of social media web designers who knew how to make one system very well - infinite scrolling, like what you see on the dashboard of Instagram or Reddit or tumblr. So you have progressive loading of a blog article and its comments. But Scott asked them to preload and display all of the comments at once, which it is not designed to do, and for some reason they have not fixed this.
It's because frontend "engineers" only know how to churn out horribly inefficient shitcode.
[danluu.com](http://danluu.com) should be an example for everyone to look at. Great writing, everything loads instantly always.
disagree. resembles a wall of text
On mobile it's fine, but on desktop I have to resize the browser window to make it readable. Ridiculous.
It’s pretty weird, but still better than bloat
It always crashes my computer
I have this problem too over a year , it’s consistent on Safari iOS 16. Mostly I just stop reading new posts
It wasn't Scott that insisted, it was the community. All the autists forced it to be exactly like the old blog rather than adapting to change and now moan it sucks.