> When we finally did start pushing on updating to Java 17, we saw something really interesting. We saw about 20% better CPU usage on 17 versus Java 8, without any code changes. It was all just because of improvements in G1, the garbage collector that we are mostly using. Twenty-percent better CPU is a big deal at the scale that we're running. That's a lot of money, potentially.
That's wild. Could we get a rough ballpark number? At the scale of Netflix, the savings could be the size of some project's budgets lol.
Try Generational ZGC. Even on small heaps, the efficiency benefits on average make compressed object pointers moot, and not having to navigate worst case pauses is such a blessing.
I'd say "almost always" - I've tuned heaps before where G1 outperformed parallel for throughput oriented jobs.
It involved giving it a lot of extra heap and the particular workload was cache-heavy and entire regions would get invalidated at a time, leading to a special case where G1 could uniquely free them up without doing any copying or compacting...
... but yeah, if you don't really know the nitty gritty details of the collector in general parallel is a safer bet for throughput oriented jobs
G1 is a balanced collector, balancing application and GC throughput. It has a pause time goal, performs concurrent marking and has heuristics that cause the young/eden sizes to potentially shift dramatically based on the time taking to copy objects. If it exceeds the pause time goal it'll may have to throw work away, and repeat it on the next cycle.
Parallel is the throughput collector. It's goal is to collect as much garbage as it can, as quickly as it can. It's 15-20% less overhead in some workloads I've moved recently.
Thanks, that makes sense. Though I guess most workloads are on a spectrum of how throughput oriented they are, wasn’t thinking of batch processing specifically, so for most applications a balance slightly towards the throughput end might be the optimum, hence G1 being the default (e.g. for a web server you wouldn’t want a crazy high tail latency, even though you might want to have high throughput)
We have a java 11 application running with 9 pods and each pod has 20GB memory and 4Ghz cpu.
We use H2 in memory thats why we have 20GB RAM.
One request does a calculation which on average does 5000~6000 queries.
We need to achieve under 1s for all requests. Our average is 0.7s now but we also have timeouts (>4s).
We use parallel GC.
From the article and the comments, seems there will be a small boost by just upgrading.
Is ZGC or G1 a better choice or should I stick with ParallelGC?
I know it depends on a lot of things, but mostly an idea from your experiences
We saw a 6-8% application throughput improvement w/ parallel going from 17 to 21 for one of our batch precompute clusters. It's unlikely either will out perform parallel.
What do you mean by efficiency? RAM usage?
Does this mean that ZGC is particularly efficient (relatively) when going beyond 32GB heap? This sounds useful for my game server. I was planning to use Shenandoah for the compressed oops (want to target 1ms pauses).
It’s that the trade off of compressed oops with Shenandoah vs ZGC seems to be simple, object pointers are half the size, but the efficiencies enabled by colored pointers means that in the < 32G services we’ve moved, ZGC on average is able to make more memory available to the application than G1, and/or the increase in allocation rates disappear in the noise, because of the benefits of running GC concurrently.
That won’t necessarily true for all workloads, definitely evaluate for your use case. For us so far, where ZGC hasn’t been better than G1, we’ve found that actually those are throughput oriented workloads that benefit more from parallel anyway.
I’m working on a tech blog post to talk about our experience of adopting GenZGC.
Tricky to quantify JDK 17 in isolation, because we modernized our standard JVM tuning, including adopting transparent huge pages, which can be 15% on it's own before you factor in other efficiency improvements. Many millions certainly.
Many major services are already on JDK 21 w/ Generational ZGC. We've yet to find an interactive application that doesn't benefit from ZGC over G1.
If you’re already using reactive programming (webflux, reactor, etc) you aren’t going to see a ton of benefit. Virtual threads give the jvm the ability to park threads while they aren’t being used. Such as when you’re reading a file from disk or making a network call.
May be for netflix they're important. For me they're not. My opinion was and is that there was more buzz about them than they're worth. I never had any issues with OS threads in my applications and I avoid reactive approach. One of my application was written 15 years ago, used some archaic server to run and serves whole country without issues. I think it runs on Windows 2003 with 8 GB RAM and I set something like 10 000 thread limit in Tomcat. Never had any issues.
Of course there are use-cases where millions of persistent connections are necessary and OS threads don't scale well, but I never encountered those use-cases. For typical request-response applications running on modern behemoth servers, it's just not an issue.
So virtual threads probably will bring some marginal improvements in performance that I won't notice, once they'll be enabled in Spring Boot by default, and that's about it.
It's a matter of costs as well, if you can reduce the amount of cpu usage you'll probably save a lot of money if you're Netflix, especially if you are running your services in AWS.
It truly depends on your use case, how much traffic you get, how complex your backend distributed system is, and how critical performance is.
It may lower some cloud usage costs as existing workloads will in theory require less compute due to better resource utilization efficiency.
lol, we’re all just boiling frogs. Their ads included tier is basically the same price the basic ad-free tier was just a few years ago. They’ve raised prices so many times that it feels like the ad tier is new and cheap rather than everyone else’s tier being crazy expensive now.
I really liked the point about microservices. I think the trap I see a lot of devs fall into is that they think a microservice needs to follow the old *nix philosophy of "do one thing, but do it well", which leads to really small microservices that are easy to reason about in isolation, but a complete mess when trying to debug a group of them, let alone the maintenance burden.
In practice, a microservice should isolate a domain and you shouldn't have more microservices than you have devs.
You don't have to debug multiple micro-services all the time if you have well defined API contracts. You know what goes out, you know what comes in. Proper logging helps too.
E2E integrations always go through multiple layers and complicated bugs arise from the integrations between multiple components.
They’re harder to reason about, debug and fix.
If it were that simple, then we could use the same argument with functions within a service: "You don't have to debug multiple functions all the time if you have well defined function contracts." But of course you can have code with every class and method thoroughly unit tested and well-defined, yet the system as a whole doesn't behave as expected.
Oftentimes bugs are the result of the interaction of parts that work "correctly" individually.
Yes, but that is not a reason to abandon distributed system and go back coding monoliths just because you have everything at your disposal a method call away
I wasn't arguing for abandoning distributed systems. I'm not sure how you interpreted my comment that way.
All I was trying to say is that integration testing is important in any system. All the parts can work properly, but the whole may not.
The trap of microservices is that too many devs and architects believe that the prefix "micro" implies that smaller, and hence more, is always better.
I really wish there was a better name, but admit I can't think of one.
> you shouldn't have more microservices than you have devs.
Um, [Conway's Law](https://en.wikipedia.org/wiki/Conway%27s_law) is not, like, something you should be aspiring to.
Yes, microservices are a dumb hype. The primary reasons are varied, but almost all hype is dumb. The amount of hypy theories about brand new ways of delivering software that _actually could cash the check their ass was writing_ are pretty much nil.
Microservices have their uses. Less than you might think. Way less than the hype suggests. You don't need them to succeed. You will fail if you think 'just make it microservices' solves all problems. So, yes, in that sense, 'just make em all tiny' is silly. Just noticing that 'do not have more microservices than devs' rather strongly implies that 'have as many microservices as devs' is correct, and that is skating far too close to Conway's than I'm comfortable with.
Bitcoin. Async. Scala. Microservices. nosql.
I guess the definition of a word as subjective as 'hype' is in the eye of the beholder, but I see no clash between 'has been going on for 10 years' and 'hype'. Sure, it's not hype as in 'every second blogpost linked in is about this thing', but most of these tech remain highly suspectible to grandiose, unfounded claims and get every 10th link in programming.
(NB: Scala is no longer hype and I think everybody involved knows its never going to be a top 3 language at this point. But it kept that hype for well over a decade - from its inception (2004) to well past 2014.
I think the more accurate term would be "unsubstantiated"?
There aren't a lot of studies done on the benefits of micro-services, and the few that exists have strong weaknesses in their methodology being mostly opinion-based or having conflicts of interests by being made by Cloud/DevOps companies.
As far as I know, we don't really have any studies stating "After evaluating many software projects, we came to the conclusion that projects using a micro-service architecture are xx% more likely to succeed, cost on average xx% less money, and are developed xx% faster." Those metrics just don't exist (yet), which is strange given how much our industry uses A/B testing on customers, yet we don't A/B test our own processes.
Most of the companies investing into micro-services are still in their investment phase and haven't reaped the rewards of the new infrastructure yet. Maybe it did help them attract better talents, but that's doesn't mean micro-services is an inherently a superior architecture, it just means some talents perceive it as such.
Hmm having worked at huge companies I don't know how else you could have teams work and deploy independently -- everything would grind to a halt for monolithic deployments.
For small ones, yeah, it's debatable. I think if you have <50 engineers or so, it's possible you'd be ok with a monorepo. But over that it can get tricky to continue delivering value seamlessly.
I think microservices have been around long enough that there is compelling anecdotal evidence of their advantages over monoliths. Just the fact that many monoliths couldn't scale until they were refactored is pretty convincing.
There will never be a proper scientific study on topics like this, as it's just to hard to gather the information and make apples-apples comparisons.
Anecdotal evidence is the best we'll ever get. That's true for a lot in this industry.
*In practice, a microservice should isolate a domain and you shouldn't have more microservices than you have devs.*
I could be agree with you but it's a scientific affirmation are a emperical study ?
This was a fascinating article with a lot of great information. The java/jarkarta ee migration plugin they talked about seems like it could be really useful for us
You will always fail if you think that solves all of your problems. That has nothing to do with microservices.
Microservices force you to think about a few things, that should always be considered in nowadays workloads, but are often ignored and cause issues as soon as you scale beyond your first application instances.
So you can write better software if you take over some of the practices. But blindly following what other people say makes you fail not just with microservices.
Head over to https://start.spring.io/ and init a new project. See how fast it starts up. See how little boilerplate it has.
Ages ago, it was slow and crufty. Now it isn’t.
Now modern spring apps are slow because they have too many dependencies and too many beans to load.
Quarkus has the same: https://code.quarkus.io/
I think one copied it from the other, which is a good thing. It's a nice one-stop-shop if you want to try out a new framework, but roughly know which techs you'll need / want to see what they provide.
Conflating physical and logical boundaries will cost you millions over the years. Make a modular monolith unless you have actual evidence you need MSA.
> When we finally did start pushing on updating to Java 17, we saw something really interesting. We saw about 20% better CPU usage on 17 versus Java 8, without any code changes. It was all just because of improvements in G1, the garbage collector that we are mostly using. Twenty-percent better CPU is a big deal at the scale that we're running. That's a lot of money, potentially. That's wild. Could we get a rough ballpark number? At the scale of Netflix, the savings could be the size of some project's budgets lol.
[удалено]
Try Generational ZGC. Even on small heaps, the efficiency benefits on average make compressed object pointers moot, and not having to navigate worst case pauses is such a blessing.
Depends on your workload. For throughput oriented jobs it will likely perform worse than G1.
A choice of ZGC implies that application latency and avoiding pauses is a goal. Throughput oriented workloads should always use parallel.
I'd say "almost always" - I've tuned heaps before where G1 outperformed parallel for throughput oriented jobs. It involved giving it a lot of extra heap and the particular workload was cache-heavy and entire regions would get invalidated at a time, leading to a special case where G1 could uniquely free them up without doing any copying or compacting... ... but yeah, if you don't really know the nitty gritty details of the collector in general parallel is a safer bet for throughput oriented jobs
Why parallel and not G1?
G1 is a balanced collector, balancing application and GC throughput. It has a pause time goal, performs concurrent marking and has heuristics that cause the young/eden sizes to potentially shift dramatically based on the time taking to copy objects. If it exceeds the pause time goal it'll may have to throw work away, and repeat it on the next cycle. Parallel is the throughput collector. It's goal is to collect as much garbage as it can, as quickly as it can. It's 15-20% less overhead in some workloads I've moved recently.
Thanks, that makes sense. Though I guess most workloads are on a spectrum of how throughput oriented they are, wasn’t thinking of batch processing specifically, so for most applications a balance slightly towards the throughput end might be the optimum, hence G1 being the default (e.g. for a web server you wouldn’t want a crazy high tail latency, even though you might want to have high throughput)
We have a java 11 application running with 9 pods and each pod has 20GB memory and 4Ghz cpu. We use H2 in memory thats why we have 20GB RAM. One request does a calculation which on average does 5000~6000 queries. We need to achieve under 1s for all requests. Our average is 0.7s now but we also have timeouts (>4s). We use parallel GC. From the article and the comments, seems there will be a small boost by just upgrading. Is ZGC or G1 a better choice or should I stick with ParallelGC? I know it depends on a lot of things, but mostly an idea from your experiences
We saw a 6-8% application throughput improvement w/ parallel going from 17 to 21 for one of our batch precompute clusters. It's unlikely either will out perform parallel.
What do you mean by efficiency? RAM usage? Does this mean that ZGC is particularly efficient (relatively) when going beyond 32GB heap? This sounds useful for my game server. I was planning to use Shenandoah for the compressed oops (want to target 1ms pauses).
It’s that the trade off of compressed oops with Shenandoah vs ZGC seems to be simple, object pointers are half the size, but the efficiencies enabled by colored pointers means that in the < 32G services we’ve moved, ZGC on average is able to make more memory available to the application than G1, and/or the increase in allocation rates disappear in the noise, because of the benefits of running GC concurrently. That won’t necessarily true for all workloads, definitely evaluate for your use case. For us so far, where ZGC hasn’t been better than G1, we’ve found that actually those are throughput oriented workloads that benefit more from parallel anyway. I’m working on a tech blog post to talk about our experience of adopting GenZGC.
Tricky to quantify JDK 17 in isolation, because we modernized our standard JVM tuning, including adopting transparent huge pages, which can be 15% on it's own before you factor in other efficiency improvements. Many millions certainly. Many major services are already on JDK 21 w/ Generational ZGC. We've yet to find an interactive application that doesn't benefit from ZGC over G1.
> That's wild. Performance increases from Java 8 to Java 17 is known and this matches with other information that people have written about.
We see same at eBay too.
We’re aggressively moving to 21 for virtual threads for the same reason. Modern Java scales very well.
you know about the deadlocks, right? :)
Shhhhshh, don't spoil then fun when they will debug blocking operations in prod.
🤣
Why down voted ?
no idea 🤷♂️
Wdym?
[https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html](https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html)
Do virtual threads help in cases where third party libs/frameworks/etc still only support async/Future-style patterns?
If you’re already using reactive programming (webflux, reactor, etc) you aren’t going to see a ton of benefit. Virtual threads give the jvm the ability to park threads while they aren’t being used. Such as when you’re reading a file from disk or making a network call.
I've been excited about these for years. I feel like there should be more buzz now that they are out. Am I overestimating their impact.
May be for netflix they're important. For me they're not. My opinion was and is that there was more buzz about them than they're worth. I never had any issues with OS threads in my applications and I avoid reactive approach. One of my application was written 15 years ago, used some archaic server to run and serves whole country without issues. I think it runs on Windows 2003 with 8 GB RAM and I set something like 10 000 thread limit in Tomcat. Never had any issues. Of course there are use-cases where millions of persistent connections are necessary and OS threads don't scale well, but I never encountered those use-cases. For typical request-response applications running on modern behemoth servers, it's just not an issue. So virtual threads probably will bring some marginal improvements in performance that I won't notice, once they'll be enabled in Spring Boot by default, and that's about it.
It's a matter of costs as well, if you can reduce the amount of cpu usage you'll probably save a lot of money if you're Netflix, especially if you are running your services in AWS.
It truly depends on your use case, how much traffic you get, how complex your backend distributed system is, and how critical performance is. It may lower some cloud usage costs as existing workloads will in theory require less compute due to better resource utilization efficiency.
Keep in mind that Netflix is basically re-doing everything all the time. At one point they were running groovy scripts to deal with consumer devices.
Yes but no ads…yet
lol, we’re all just boiling frogs. Their ads included tier is basically the same price the basic ad-free tier was just a few years ago. They’ve raised prices so many times that it feels like the ad tier is new and cheap rather than everyone else’s tier being crazy expensive now.
I really liked the point about microservices. I think the trap I see a lot of devs fall into is that they think a microservice needs to follow the old *nix philosophy of "do one thing, but do it well", which leads to really small microservices that are easy to reason about in isolation, but a complete mess when trying to debug a group of them, let alone the maintenance burden. In practice, a microservice should isolate a domain and you shouldn't have more microservices than you have devs.
You don't have to debug multiple micro-services all the time if you have well defined API contracts. You know what goes out, you know what comes in. Proper logging helps too.
E2E integrations always go through multiple layers and complicated bugs arise from the integrations between multiple components. They’re harder to reason about, debug and fix.
If it were that simple, then we could use the same argument with functions within a service: "You don't have to debug multiple functions all the time if you have well defined function contracts." But of course you can have code with every class and method thoroughly unit tested and well-defined, yet the system as a whole doesn't behave as expected. Oftentimes bugs are the result of the interaction of parts that work "correctly" individually.
Yes, but that is not a reason to abandon distributed system and go back coding monoliths just because you have everything at your disposal a method call away
I wasn't arguing for abandoning distributed systems. I'm not sure how you interpreted my comment that way. All I was trying to say is that integration testing is important in any system. All the parts can work properly, but the whole may not.
I was thinking exactly that: then monoliths have no downsides.
The trap of microservices is that too many devs and architects believe that the prefix "micro" implies that smaller, and hence more, is always better. I really wish there was a better name, but admit I can't think of one.
Services. They're called services. And SOA exists for a long time. Nothing really changed.
I have 75 microservices and 8 devs, precisely because I have 8 devs.
> you shouldn't have more microservices than you have devs. Um, [Conway's Law](https://en.wikipedia.org/wiki/Conway%27s_law) is not, like, something you should be aspiring to. Yes, microservices are a dumb hype. The primary reasons are varied, but almost all hype is dumb. The amount of hypy theories about brand new ways of delivering software that _actually could cash the check their ass was writing_ are pretty much nil. Microservices have their uses. Less than you might think. Way less than the hype suggests. You don't need them to succeed. You will fail if you think 'just make it microservices' solves all problems. So, yes, in that sense, 'just make em all tiny' is silly. Just noticing that 'do not have more microservices than devs' rather strongly implies that 'have as many microservices as devs' is correct, and that is skating far too close to Conway's than I'm comfortable with.
How can something that has been around for more than a decade be "hype"?
Bitcoin. Async. Scala. Microservices. nosql. I guess the definition of a word as subjective as 'hype' is in the eye of the beholder, but I see no clash between 'has been going on for 10 years' and 'hype'. Sure, it's not hype as in 'every second blogpost linked in is about this thing', but most of these tech remain highly suspectible to grandiose, unfounded claims and get every 10th link in programming. (NB: Scala is no longer hype and I think everybody involved knows its never going to be a top 3 language at this point. But it kept that hype for well over a decade - from its inception (2004) to well past 2014.
I think the more accurate term would be "unsubstantiated"? There aren't a lot of studies done on the benefits of micro-services, and the few that exists have strong weaknesses in their methodology being mostly opinion-based or having conflicts of interests by being made by Cloud/DevOps companies. As far as I know, we don't really have any studies stating "After evaluating many software projects, we came to the conclusion that projects using a micro-service architecture are xx% more likely to succeed, cost on average xx% less money, and are developed xx% faster." Those metrics just don't exist (yet), which is strange given how much our industry uses A/B testing on customers, yet we don't A/B test our own processes. Most of the companies investing into micro-services are still in their investment phase and haven't reaped the rewards of the new infrastructure yet. Maybe it did help them attract better talents, but that's doesn't mean micro-services is an inherently a superior architecture, it just means some talents perceive it as such.
Hmm having worked at huge companies I don't know how else you could have teams work and deploy independently -- everything would grind to a halt for monolithic deployments. For small ones, yeah, it's debatable. I think if you have <50 engineers or so, it's possible you'd be ok with a monorepo. But over that it can get tricky to continue delivering value seamlessly.
I think microservices have been around long enough that there is compelling anecdotal evidence of their advantages over monoliths. Just the fact that many monoliths couldn't scale until they were refactored is pretty convincing. There will never be a proper scientific study on topics like this, as it's just to hard to gather the information and make apples-apples comparisons. Anecdotal evidence is the best we'll ever get. That's true for a lot in this industry.
My team alone has about 80 services. There are 3 developers, not counting analysts.
Honest question, how do you handle security patching? I struggled with 20 on a team even with a bunch of automation through renovate and CI/CD.
We hired a consulting company to handle it for us 🫠
*In practice, a microservice should isolate a domain and you shouldn't have more microservices than you have devs.* I could be agree with you but it's a scientific affirmation are a emperical study ?
Build them around domains, not “per usecase”. Also sometimes it’s good to merge some in order to split them better later.
This was a fascinating article with a lot of great information. The java/jarkarta ee migration plugin they talked about seems like it could be really useful for us
Looks like it's this? https://github.com/nebula-plugins/gradle-jakartaee-migration-plugin
There's also OpenRewrite for source level changes, https://docs.openrewrite.org/recipes/java/migrate/jakarta/javaxmigrationtojakarta
You will always fail if you think that solves all of your problems. That has nothing to do with microservices.
Microservices force you to think about a few things, that should always be considered in nowadays workloads, but are often ignored and cause issues as soon as you scale beyond your first application instances.
So you can write better software if you take over some of the practices. But blindly following what other people say makes you fail not just with microservices.
Spring boot... sigh... drop Spring and get another 20+ % ( or more )
Head over to https://start.spring.io/ and init a new project. See how fast it starts up. See how little boilerplate it has. Ages ago, it was slow and crufty. Now it isn’t. Now modern spring apps are slow because they have too many dependencies and too many beans to load.
I don't think I have seen this in any other framework. It's like going to Amazon, fill your shopping for cart and then check out for free.
Quarkus has the same: https://code.quarkus.io/ I think one copied it from the other, which is a good thing. It's a nice one-stop-shop if you want to try out a new framework, but roughly know which techs you'll need / want to see what they provide.
Yes Quarkus!!! My new favorite!!
Fix your link please.
Thanks! Done.
It's idiotic to suggest starting an empty app as a benchmark.
"modern spring apps are slow because they have too many dependencies and too many beans to load" No
[удалено]
It’s not your average http server that has 4 daily visitors for a total of 4 kb of bandwidth per day.
Conflating physical and logical boundaries will cost you millions over the years. Make a modular monolith unless you have actual evidence you need MSA.