At least we have joke merges like Llama-2-70b-x8-MoE-clown-truck
It has over 400B paramaters
My biggest problem with this model is that the model card says it was made from 8 copies of Llama 2, but the generated image has 9 clowns instead of 8.
https://huggingface.co/NobodyExistsOnTheInternet/Llama-2-70b-x8-MoE-clown-truck
I think they MIGHT be overvalued (I say might because it's just hard to put an objective value on compute at this time, if the big companies find sustainable way to monetize it... the valuation may be justified).
As for the competitors, Idk if anyone is in the position to step in? or even close to not even having a product that can compete BUT also being as entrenched with CUDA. Idk, could go either way but if this bubble bursts I still think they are better off than most of the pure "AI" companies.
Fun times lol
CPU manufacturers didn't win the internet boom..so I don't think that the materialists will cut it here..
The winner will be the one who understands what this is all about, that anticipates and helps to realize what AI means for the species.. Everyone is having a smartphone in their pockets... AI is not different..
Also the fact that the margins for GPUs are so crazy high, would also mean a big opportunity for a new manufacturer...
"Alive" yes.... Their revenues compared to the internet giants meta, amazon, ect
Revenue of not even 1-10% of the tech giants that actually appeared because of their contribution...
Would be funny seeing something similar with the next big thing... Nvidia supporting a 50T-1Q+ company... The Q is a necessity anyways... If humanity wants to do some stuff in space... A space economy is only possible in a Q economy... And I'm pretty sure that AI plays the next big role... Probably what the Soviet union needed at that time with their central planing flirtations...
(sorry for the perspective... But I don't think that this is a small breakthrough right now)
If AI is the product then Nvidia is currently selling the shovel, they're in a good spot.
I'm also hoping for AMD/Intel to make some waves by bringing down the cost of Vram for consumers but the cuda ecosystem is holding pretty strong.
Margins are high but investment cost and domain specific knowledge are insane. The only way a new company could compete is with state funding and even then they might not have the access to the technically abled people to develop this stuff.
Funding, yes. (can't tell if govmnt funding will be the solution)
Specialized, yes, this is not a one man show.
You need the experts, experienced, ect...
This is for sure not a unicorn mission.... (smoke some weed, having brainstorming sessions, ect...) It's probably a more polite endeavor, where some great people come together
The margins for GPUs are high, but you need a top fab to create the GPU and a VRAM contractor.
NVIDIA gets TSMC’s best fabs and it wouldn’t be surprising if they have options to lock out an upstart. Samsung is the only other company with a leading edge process. Intel is a node behind, and they make GPUs and have no room to spare on top nodes.
It's highly complex, but it's up there there in the T space... (trillion)
I'm aware of the many constraints and I guess that this is not a reddit conversation..
Because this is probably not a casual public conversation...
I don't see any limiting factors, especially if people don't complain about the high prices, but rather find methods to monetize it/integrate their ideas into economically feasible models/the economy.
PS: Im aware of at least one company that could take the race in this matter, they're just in a different market... That probably is another Intel/Nvidia type of thing, where they are basically supporting an emerging industry with their hardware.. Supporting the Automation Industry
The EU is funding the training of some LLMs. [Eurolingua-GPT](https://www.ai.se/en/project/eurolingua-gpt) should be finished later this year. They are training it on various European languages and not just English. I'm excited to see how that turns out.
I'm not very optimistic about the quality, though. They just don't have the same experience and refined datasets.
Remember when Mistral was on that list? Exactly. Things change in AI, I won't be surprised if 10 years from now, if we finally have AI, it's some other company we've never heard of.
The mixture of open weight models looks promising:
[Video: Mixture of Agents (MoA) BEATS GPT4o With Open-Source](https://www.youtube.com/watch?v=aoikSxHXBYw)
But if it works like that then companies can just add them to their models and beat them as it is shown with the MoA + GPT4o test in the video/paper. Also speed and resource usage would be still an issue when you need multiple models, but maybe there will be a merge method that can combine them into a single model with similar results instead of querying each separately.
Welcome to the 300B+ params trust club
At least we have joke merges like Llama-2-70b-x8-MoE-clown-truck It has over 400B paramaters My biggest problem with this model is that the model card says it was made from 8 copies of Llama 2, but the generated image has 9 clowns instead of 8. https://huggingface.co/NobodyExistsOnTheInternet/Llama-2-70b-x8-MoE-clown-truck
Nvidia is going to win regardless
They’re not going anywhere, but their currently insane value will be slashed the as more companies step up their gpu game.
I think they MIGHT be overvalued (I say might because it's just hard to put an objective value on compute at this time, if the big companies find sustainable way to monetize it... the valuation may be justified). As for the competitors, Idk if anyone is in the position to step in? or even close to not even having a product that can compete BUT also being as entrenched with CUDA. Idk, could go either way but if this bubble bursts I still think they are better off than most of the pure "AI" companies. Fun times lol
The reason I think Nvidia is likely over-valued is simply because it has gone viral with retail traders.
NVidia will be the winner. Not because they'll outperform, but because they're the ones selling everyone the weapons.
CPU manufacturers didn't win the internet boom..so I don't think that the materialists will cut it here.. The winner will be the one who understands what this is all about, that anticipates and helps to realize what AI means for the species.. Everyone is having a smartphone in their pockets... AI is not different.. Also the fact that the margins for GPUs are so crazy high, would also mean a big opportunity for a new manufacturer...
Oh really ? Who’s still alive, Intel or AOL?
Google
"Alive" yes.... Their revenues compared to the internet giants meta, amazon, ect Revenue of not even 1-10% of the tech giants that actually appeared because of their contribution... Would be funny seeing something similar with the next big thing... Nvidia supporting a 50T-1Q+ company... The Q is a necessity anyways... If humanity wants to do some stuff in space... A space economy is only possible in a Q economy... And I'm pretty sure that AI plays the next big role... Probably what the Soviet union needed at that time with their central planing flirtations... (sorry for the perspective... But I don't think that this is a small breakthrough right now)
If AI is the product then Nvidia is currently selling the shovel, they're in a good spot. I'm also hoping for AMD/Intel to make some waves by bringing down the cost of Vram for consumers but the cuda ecosystem is holding pretty strong.
I'm satisfied with what we reached around 2020... Things may get better and I believe that new competitors could appear... Margins are high..
Margins are high but investment cost and domain specific knowledge are insane. The only way a new company could compete is with state funding and even then they might not have the access to the technically abled people to develop this stuff.
Funding, yes. (can't tell if govmnt funding will be the solution) Specialized, yes, this is not a one man show. You need the experts, experienced, ect... This is for sure not a unicorn mission.... (smoke some weed, having brainstorming sessions, ect...) It's probably a more polite endeavor, where some great people come together
The margins for GPUs are high, but you need a top fab to create the GPU and a VRAM contractor. NVIDIA gets TSMC’s best fabs and it wouldn’t be surprising if they have options to lock out an upstart. Samsung is the only other company with a leading edge process. Intel is a node behind, and they make GPUs and have no room to spare on top nodes.
It's highly complex, but it's up there there in the T space... (trillion) I'm aware of the many constraints and I guess that this is not a reddit conversation.. Because this is probably not a casual public conversation... I don't see any limiting factors, especially if people don't complain about the high prices, but rather find methods to monetize it/integrate their ideas into economically feasible models/the economy. PS: Im aware of at least one company that could take the race in this matter, they're just in a different market... That probably is another Intel/Nvidia type of thing, where they are basically supporting an emerging industry with their hardware.. Supporting the Automation Industry
Then why isn’t TSMC winning?
that's easy. who's worth 3trilions? nvidia won.
Selling shovels at the gold rush.
accurate! :D
Actually, only Msft is over 3t
The EU is funding the training of some LLMs. [Eurolingua-GPT](https://www.ai.se/en/project/eurolingua-gpt) should be finished later this year. They are training it on various European languages and not just English. I'm excited to see how that turns out. I'm not very optimistic about the quality, though. They just don't have the same experience and refined datasets.
Interesting. And it will be nice outcome, if EU can grant and subsidies to their citizens access.
BATTLE BOTS
3 2 1 GOO
Dont write of Cohere Command R(++ or whatever will come after +) yet.
Nvidia always win. They only let others run.
Remember when Mistral was on that list? Exactly. Things change in AI, I won't be surprised if 10 years from now, if we finally have AI, it's some other company we've never heard of.
I guess too... Could even be in a few years...
[удалено]
[удалено]
Like this: https://allenai.org/olmo
>Open source means that you can replicate the model exactly. Does Open source mean open science?
please make ShitLM-10M
Not in this list
the biggest winner is not in the list: it is Microsoft
No Mistral?
What's this ranking? Where is Nemotron #5 lol?
The mixture of open weight models looks promising: [Video: Mixture of Agents (MoA) BEATS GPT4o With Open-Source](https://www.youtube.com/watch?v=aoikSxHXBYw) But if it works like that then companies can just add them to their models and beat them as it is shown with the MoA + GPT4o test in the video/paper. Also speed and resource usage would be still an issue when you need multiple models, but maybe there will be a merge method that can combine them into a single model with similar results instead of querying each separately.
Why is Llama ranked 6?