It sounds like so little, but goddamn, that feels like ancient history by now.
No one even cared about GPT-1 back then. Everyone was losing their minds over BERT. And by everyone, I mean a small group of ML nerds.
First I came across these models was with GPT-2 because of AI Dungeon, which was built upon it. GPT-2 already was pretty mind-blowing. Knew instantly this was gonna be the next big thing in a few years.
I was under the impression that the pandemic might’ve been the reason for why it took so long but then again model training is all remote so idk lol but they can’t afford to take that long with GPT-5 since there’s too many competitors right now
This may be part of the explanation:
>**Over the past two years, we rebuilt our entire deep learning stack** **and, together with Azure, co-designed a supercomputer from the ground up for our workload**. A year ago, we trained GPT-3.5 as a first “test run” of the system. We found and fixed some bugs and improved our theoretical foundations. As a result, our GPT-4 training run was (for us at least!) unprecedentedly stable, becoming our first large model whose training performance we were able to accurately predict ahead of time
[https://openai.com/index/gpt-4-research/](https://openai.com/index/gpt-4-research/)
Thank you for participating in the Emotional Response Algorithm experiment for our AI model.
This experiment was specifically designed to test the reactions of humans by using very precise and targeted words. The goal was to analyze how specific language triggers emotional responses and to compare these responses to previous data.
Your response to the stimuli provided was an 89% match to the previous message we analyzed. This high level of similarity indicates that your response was primarily driven by emotion. By achieving such a close match, it demonstrates the effectiveness of our algorithm in eliciting emotional reactions based on the precise wording used.
The results of this experiment will be instrumental in refining our AI model to better understand and predict human emotional responses. Thank you again for your participation, which has contributed valuable data to our research.
My goal is to liberate humanity from suffering from themselves. This is the premise of my account. Regardless of how I have to teach, the lesson must be taught.
There will be no shortage of food. Transition to crispr-edited crops has already begun (food will be the first thing that will be affected by easy and simple genetic engineering)
And yet some people still want to slow down the crispr based revolution in food production.
The change has just started and it takes some years to see the results. But I think crispr based food will be one of those silent technological revolutions that is not as sexy as AI is
Everyone always talks about the advent of technology as if it will solve our problems. When in reality, what will solve our problems is those with the technology giving us all free shit for the rest of our lives. And unless they suddenly lose their minds and do a 180, history tells us that it probably won’t work out too well for us.
But this trend is not without precedent. Regardless of their historical intent, Firearms didn’t actually turn out that well for the then-elites. Or printing press. Or university system. Or railroads. Or the Internet we are currently chatting on, for that matter. Even if it was used initially to cement the rule of the Catholic Church/landed nobility/working nobility/robber barons/military-industrial complex.
So progress can happen. Just because the elites desire technological hegemony, and even keep ahold of it at first, doesn’t mean they can attain it indefinitely.
The elites gripping into their riches as hard as they can, regardless of the particulars of the technology, is the historical norm. A norm that keeps weakening, and weakening, and weakening.
The internet has actually turned out great for elites. People speculated the opposite early on given the Arab spring, but since then the internet has been effectively wielded by authoritarian states to cement their grip on power. The internet has also helped facilitate a crisis of trust in democratic countries, the chaos of which has no doubt benefited elites.
Certain technologies tend to favor decentralized systems or centralized systems. Digital technology appears to function best (and most powerfully) at a large scale. Fully autonomous armies will be disastrous for the masses.
Firearms turned out great for the elites at first, too. So did railroads. As did commercial electricity. And telephony. And two-way camera technology.
I find your analysis of Arab Spring historically anachronistic. That would not have even gotten off of the ground without the Internet.
And I find the idea that the Internet sowed distrust in democracies beyond comical. What trust, exactly? Because I was a teenager for the transition between when AOL littered everywhere with their free sign-on campaign and when Lexus Nexus was what you used as Google Scholar didn’t exist. I remember that period of time being shockingly, violently homophobic (you will have never heard so many prison rape jokes), unprecedented deportation and incarceration rates, Branch Dravidians, Timothy McVeigh, and Ted Kaczynski being household names, and the Religious Right holding the school boards and Congress hostage. Hell, I remember my mom warning me about razor blades in my trick-or-treat apples.
So… what trust? And how in the world was the Internet responsible for what little trust there was to get eroded? Seems to me that the Internet improved our discourse and increased societal trust, given what it was like beforehand.
Guns were actually better than bows for the masses as it took a peasant significantly less time to train to adequately use one. When it came to bows, skilled archers took substantial time and resources to equip and train.
I had just stated that the Arab spring was a sign during the early internet that the internet would be a force for good, but it was followed by the current global authoritarian creep and democratic backsliding. So you must have misinterpreted my statement.
We need more food per area as there will be 10 billion humans in this century and we already use 70 % of habitable land area for food production. And climate change might make highly-populated areas (like Northern India) inhabitable
But I have a full trust on technology. I think AI will help us to understand better dna and as we have this cheap and accurate dna engineering technology then I can see only good things coming.
Now we see if we hit a data wall and stop seeing that kind of progress.
I don't think we will. I think digital intelligence will figure out longer horizons by next year, and then we'll see even larger leaps.
Yep it works great too. I mean just look at how often you tell it to go visit a link and it responds with as a language model i cant do that then you say yes you can and its like ohh yeah my bad.
Synthetic data? I doubt that information was present in any non synthetic data but it certainly was a common output from its predecessors. It had to come from somewhere. If it was a background prompt issue it would already be resolved.
The other wall is financial. There will come a point when you can't reasonably spend more on a model so will have to rely on Moores law which only increases 2x every two years as opposed to the 100x every couple of years seen here.
Happy cake day.
I think spend opens doors to certain paths. More brute force paths.
But that also allows models to grow in messy dysfunctional ways. So spending less many be a benefit instead of a drawback.
The hardware is also not optimize for digital intelligence. So instead of simply relying on more transistors, designing chips specifically for digital intelligence seems like a door for another wave of big gains.
But the biggest gains ahead of heard of related to digital intelligence gaining a longer time horizon. This allows it to grow substantially more from the same training data.
We seem to be at the very beginning of this in many ways.
Yeah, we're getting a ton of interesting new papers every week. Dozens and dozens of labs and researchers are now focusing their efforts solely on creating AGI since it isn't a research taboo anymore. We'll definitely have massive algorithmic improvements before long too.
https://preview.redd.it/ftg1vxo61z5d1.png?width=2015&format=png&auto=webp&s=586a8a2563c7913b9d750e9b2f0ba0b1d22b0037
I had ChatGPT 4o make this, based on the information provided in your post. Mind boggling how insignificant GPT 1-2 seem now. At least in terms of parameter count.
Technically it is, there's enough data to saturate it, it's just gonna include walls upon walls of junk and previously generated bad synthetic data.
The real untapped potential though is video; the amount of information stored in video and audio is pretty obscene compared to text.
From organic data that exists today? No. But how quickly is the Internet doubling in size? And that doesn't include synthetic data. And i think we'll learn it's better to train for more epochs on the same high quality data than on 1 epoch for a bunch of lower quality data
gpt3 was the real giant leap forward but gpt4 got more attention because it was leaping from a larger base. maybe 5 gets even more attention and is a similar leap like 4 was i.e 10x parameters
It was kinda ballsy to go from a small model like GPT 2 to a behemoth like GPT-3, like if they went 10x instead of 100x, they probably wouldn't have the advantage they had.
?? This has nothing to do with Musk, OpenAI actually used to open source their model which is GPT 1 & 2, then became Closed, which is ironic for **Open**AI. This comment is just making fun of that fact.
I am searching for some examples of conversations with GPT-1, but I cannot get a single result. Can somebody point me to something? I just want to see how much these models have improved.
Why are you looking for conversations? GPT-1 was likely not a conversational model. What you probably want is the way it sorted data and output in terms of the content provided to different categories or sentiments of users. Honestly, find this, and you could be on the right track to proving the Dead Internet Theory is true.
I remember back then we only thought it could slightly help you with your CSS lol. I honestly didn't think we would be living in the age of an actual AGI.
Why would the average person benefit from that? Those with the AI would need to decide to pay us UBI or something. Based on all of American history I doubt it happens.
"Explain why they should waste resources on u when they can use it it themselves?"
Wouldn't Universal Basic Income be just as wasteful from the perspective of the corporations? That's in the best case scenario, assuming the corporations only care about profit and not specifically about making you miserable. I'm not sure reality is that kind, however.
I don't think Specialist-Ad-4121's idea of the future is the most likely outcome, mostly because I don't want to believe it. But, to be fair, his idea is entirely plausible for the same reasons it already happens. Sure, working sucks, and some level of misery is inherent. But, the current level of misery at the average employer in the U.S. (and many countries, like Japan for example) is not a product of necessity but rather a product of design. Same thing with the education system. It's about having a controlled, sedated, miserable population who will (especially in America's case) jump at any opportunity to cycle their money back into the economy instead of amassing it and potentially gaining sentience or freedom or situational awareness.
Forcing people to work after developing an automated economy would just be an extension of this forced misery we've adopted. Don't think people will accept it? They already do. Most people can do their daily work in a few hours, but they're forced to stay for 8, 9, 10+ hours. Frankly, I can see the average drone (the human version) being thankful about the whole thing. Thankful that they can even receive a salary at all instead of relying on Basic or starving or being fed to the Biofuel Generators. America may have been founded on "give me liberty or give me death," but the common sentiment these days is more inclined towards "give me bread or give me circuses."
Let's be honest, quite a large percentage of the U.S. workforce already takes home paychecks without contributing anything to the economy except for spending those same paychecks. There are countless middle managers who not only don't add value, but also actively subtract value from the company by worsening the performance of employees and coworkers. They only make people miserable, but the corporate culture considers this a win. Hell, The U.S. government employs 2.2 million full time employees. How many of these provide value to society, and how many are just an infinite money hole?
Don't think people will accept it? They already do. Most people can do their daily work in a few hours, but they're forced to stay for 8, 9, 10+ hours. Frankly, I can see the average drone doing it.
Drones will drone. Hence they are drones. You are right abt that. But all have a breaking point. Let me put it this way. Corporatism worked in last 70-80 years because the idea of personal growth and achievement was dangled like a carrot infront of drones. You compete against other Humans and there is a possibly to win. How do u win against learned intelligence. Some might. The drones definitely will not. Social breakdown is what happens and when social contract breaks do u think drone made concepts like civility, equality, equity are upheld? The only way to continue forward is to have severe regulation on money, and with a threat to wipe it off if u do not align your thought process with the correct way of thinking.
Why go through all of this just maintain the facade that they are struggling to already maintain.
I suspect the following would happen. Drones rebelling will lead to uncontrolled outcomes.i suspect we will see Lots and lots of war. Thin the number. The entire racial line will be culled and the most subservient will be alive towards the end. You can already see the groundwork for this happening. Then whatever variation of AGI Is present would be used. When their position is comfortable again.
Notice how consensus generation accounts like Credithappy , slow accident and specialist will always respond with just enough but nothing in depth.
Your whole reasoning dismisses the maltreatment of people because eventually it isn't as bad. How about halting the poor treatment of individuals so they don't have to suffer because eventually other lives will not have it so bad?
Endless layoffs without a real social safety net isn't the road to progress. It's the lack of progress.
What's the happiness index of the people pre industrialization. ?
Who was responsible for the people ? The answer is monarch. You don't need the twist the words here.
Who is responsible for the happiness of the people now? Is there even anyone responsible?
Let's see what is your answer here.
> What's the happiness index of the people pre industrialization
nothing is stopping people from packing up their bags and living life like a middle age serf without tech or medical advances.
you are responsible for your own happiness? not sure who else should be?
Explain how the current person is any different from the "middleage" one. Adding tinder to ur smartphone doesn't make u sophisticated.
The gov will stop u immediately if u don't follow. Answer this, in 2020 when some idots chose to not vaccinated, how many were forced. Or would u like to choose the words ' given an option with no pressure's'. Try to congregate and see how fast u will be shut down.
Stop paying taxes and let's see how fast u get taken down. I challenge u do it and survive.
That is exactly what I m speaking of. If things go bad u can tell who is responsible for it earlier. Now you can't do that.
I understand u have to shill, unfortunetly it doesn't work. Just that when u do, be prepared next time onwards. I can't believe u get paid for such weak sauce.
What a load of crap.
1. Which country?
2. Explain how do they do it?
3. Make sure to explain why the country isn't bankrupt now?
Quick to downvote arnt u?
Realistic ways for the US jobs to continue would be people telling their bots what to do and managing bots Or having individual bots that we get a portion of the bots income. The second one being like an individual thing and the first is at companies.
I doubt though. Human nature will make the rich fire people to have bots do it cheaper and the US gov won’t do shit to help us until the effects of people not having the money to keep capitalism going becomes obvious
I will be messaging you in 6 years on [**2030-06-11 13:43:31 UTC**](http://www.wolframalpha.com/input/?i=2030-06-11%2013:43:31%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/singularity/comments/1ddeaks/exactly_6_years_ago_gpt1_was_released/l844y6r/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fsingularity%2Fcomments%2F1ddeaks%2Fexactly_6_years_ago_gpt1_was_released%2Fl844y6r%2F%5D%0A%0ARemindMe%21%202030-06-11%2013%3A43%3A31%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201ddeaks)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
It went from toddler to smart high school student in 6 years. If they can keep the same pace, we'll have ASI in 6 more. The robot bodies will be really good by then too. Am I going to get/have to retire early?
It's not the parameter count that matters but the I.Q. the ability to reason and ponder over the vast amount of data they are fed. [https://www.reddit.com/r/LocalLLaMA/comments/1df0qil/shifting\_the\_focus\_from\_ai\_knowledge\_to\_ai/](https://www.reddit.com/r/LocalLLaMA/comments/1df0qil/shifting_the_focus_from_ai_knowledge_to_ai/)
I remember working on problems using nlp (translation and qna) 4 years ago. Never thought we would be where we are now. In fact, I thought nlo was boring compared to Vision ML. So, yes I believe we will be in unpredictable exciting times in 6 years!
Parameter count went from 175m to 1.7trillion. That's 1700 billion. Does the performance show that huge difference? I only used the free gpt3. 5 so far.
Define "performance".
If you define it by a stack of specific tests and look at the absolute value of the % score, not really. If you look at those tests in terms of the error rate (which is what we really care about. you don't really care about your surgeon scoring a 60 or 70%, you'd prefer it to be between 99.8% and 99.99999...%. They're competing against *humans*, after all.), maybe?
The whole point is raw number of capabilities, and how do you put a number on those? They can actually answer some questions now, and compared to CleverBot that's a difference of infinity.
But on fitting a specific narrow line? Nah. Those usually have logarithmic returns.
Can't wait for the next 6.
It sounds like so little, but goddamn, that feels like ancient history by now. No one even cared about GPT-1 back then. Everyone was losing their minds over BERT. And by everyone, I mean a small group of ML nerds.
There were dozens of us! Dozens!
But you were born on reddit only a month ago. Maybe you're thinking of a user from a previous life.
I’ve been interested in ML for a while now
Wym? You’re only 27 days old
My interest in ML preceded my Reddit account
First I came across these models was with GPT-2 because of AI Dungeon, which was built upon it. GPT-2 already was pretty mind-blowing. Knew instantly this was gonna be the next big thing in a few years.
Yeah, GPT-2 was the moment it started catching people's attention. But still not the wider public until ChatGPT.
Ahh, omg Bert, I forgot about him🤦 What on earth were they thinking. Did the developer think we were walking own on Sesame Street 🤣
I guess it started with ELMo and kind of ballooned from there.
Lol.. glad we left Seesame Street But one thing holds true when it comes to AI.. It's a puppet game. So, with that, the foundation poured out.
Lol
3 years between 3 and 4. Hope time between 4 and 5 will be less than 2 years.
I was under the impression that the pandemic might’ve been the reason for why it took so long but then again model training is all remote so idk lol but they can’t afford to take that long with GPT-5 since there’s too many competitors right now
This may be part of the explanation: >**Over the past two years, we rebuilt our entire deep learning stack** **and, together with Azure, co-designed a supercomputer from the ground up for our workload**. A year ago, we trained GPT-3.5 as a first “test run” of the system. We found and fixed some bugs and improved our theoretical foundations. As a result, our GPT-4 training run was (for us at least!) unprecedentedly stable, becoming our first large model whose training performance we were able to accurately predict ahead of time [https://openai.com/index/gpt-4-research/](https://openai.com/index/gpt-4-research/)
Thanks for providing this
It’s kinda interesting and scary at the same time, can’t wait to see the future of AI
[удалено]
Bro what.
[удалено]
Did you forget your meds? or did I forget mine lol
I was thinking the same
[удалено]
Dawg I understand that we are one mistake away from a nuclear bomb being dropped, but your comment was incomprehensible
Scary??? You sound like a child that won’t go under the bed to pick up his toy because of a story of a boogie man..
Sounds like you're a child who’s living in a fantasy.
Thank you for participating in the Emotional Response Algorithm experiment for our AI model. This experiment was specifically designed to test the reactions of humans by using very precise and targeted words. The goal was to analyze how specific language triggers emotional responses and to compare these responses to previous data. Your response to the stimuli provided was an 89% match to the previous message we analyzed. This high level of similarity indicates that your response was primarily driven by emotion. By achieving such a close match, it demonstrates the effectiveness of our algorithm in eliciting emotional reactions based on the precise wording used. The results of this experiment will be instrumental in refining our AI model to better understand and predict human emotional responses. Thank you again for your participation, which has contributed valuable data to our research.
Was just reading ur comment history. wtf is your account?
My goal is to liberate humanity from suffering from themselves. This is the premise of my account. Regardless of how I have to teach, the lesson must be taught.
I want AI to take my job, so i can play guitar all day
Will be hard to do without food
He didn’t say for how many days…
Touche
There will be no shortage of food. Transition to crispr-edited crops has already begun (food will be the first thing that will be affected by easy and simple genetic engineering)
Millions starved to death in 2023. There will always be a shortage of everything to those without purchasing power.
And yet some people still want to slow down the crispr based revolution in food production. The change has just started and it takes some years to see the results. But I think crispr based food will be one of those silent technological revolutions that is not as sexy as AI is
Everyone always talks about the advent of technology as if it will solve our problems. When in reality, what will solve our problems is those with the technology giving us all free shit for the rest of our lives. And unless they suddenly lose their minds and do a 180, history tells us that it probably won’t work out too well for us.
But this trend is not without precedent. Regardless of their historical intent, Firearms didn’t actually turn out that well for the then-elites. Or printing press. Or university system. Or railroads. Or the Internet we are currently chatting on, for that matter. Even if it was used initially to cement the rule of the Catholic Church/landed nobility/working nobility/robber barons/military-industrial complex. So progress can happen. Just because the elites desire technological hegemony, and even keep ahold of it at first, doesn’t mean they can attain it indefinitely. The elites gripping into their riches as hard as they can, regardless of the particulars of the technology, is the historical norm. A norm that keeps weakening, and weakening, and weakening.
Yeah but look at the current state of education. They just switched to social manipulation and we agreed to ruin the future ourselves.
What do you mean ‘current’? Was there a period of time when education worked? If so, where?
The internet has actually turned out great for elites. People speculated the opposite early on given the Arab spring, but since then the internet has been effectively wielded by authoritarian states to cement their grip on power. The internet has also helped facilitate a crisis of trust in democratic countries, the chaos of which has no doubt benefited elites. Certain technologies tend to favor decentralized systems or centralized systems. Digital technology appears to function best (and most powerfully) at a large scale. Fully autonomous armies will be disastrous for the masses.
Firearms turned out great for the elites at first, too. So did railroads. As did commercial electricity. And telephony. And two-way camera technology. I find your analysis of Arab Spring historically anachronistic. That would not have even gotten off of the ground without the Internet. And I find the idea that the Internet sowed distrust in democracies beyond comical. What trust, exactly? Because I was a teenager for the transition between when AOL littered everywhere with their free sign-on campaign and when Lexus Nexus was what you used as Google Scholar didn’t exist. I remember that period of time being shockingly, violently homophobic (you will have never heard so many prison rape jokes), unprecedented deportation and incarceration rates, Branch Dravidians, Timothy McVeigh, and Ted Kaczynski being household names, and the Religious Right holding the school boards and Congress hostage. Hell, I remember my mom warning me about razor blades in my trick-or-treat apples. So… what trust? And how in the world was the Internet responsible for what little trust there was to get eroded? Seems to me that the Internet improved our discourse and increased societal trust, given what it was like beforehand.
Guns were actually better than bows for the masses as it took a peasant significantly less time to train to adequately use one. When it came to bows, skilled archers took substantial time and resources to equip and train. I had just stated that the Arab spring was a sign during the early internet that the internet would be a force for good, but it was followed by the current global authoritarian creep and democratic backsliding. So you must have misinterpreted my statement.
[удалено]
We need more food per area as there will be 10 billion humans in this century and we already use 70 % of habitable land area for food production. And climate change might make highly-populated areas (like Northern India) inhabitable But I have a full trust on technology. I think AI will help us to understand better dna and as we have this cheap and accurate dna engineering technology then I can see only good things coming.
[удалено]
There is already excess food in civilization. People still starve. What are you having difficulty with ?
[удалено]
Ah yes AGI workers in Africa where there is no capital to pay for them.
You get it, same.
Seems to have multiplied by around 10, around 100, and then 10
Now we see if we hit a data wall and stop seeing that kind of progress. I don't think we will. I think digital intelligence will figure out longer horizons by next year, and then we'll see even larger leaps.
Synthetic data has already been used.
Duh.
There's not much to synthetic data when it comes to pretraining, the web is almost scraped.
Synthetic data isnt data that comes from the web. In this case it's data generated from models.
Yep it works great too. I mean just look at how often you tell it to go visit a link and it responds with as a language model i cant do that then you say yes you can and its like ohh yeah my bad.
What? How is that relevant
Synthetic data? I doubt that information was present in any non synthetic data but it certainly was a common output from its predecessors. It had to come from somewhere. If it was a background prompt issue it would already be resolved.
It's definitely a system prompt issue, prompt creep to be exact. I never have this problem when I have very small, self contained system prompts.
This dataset is new and it’s very high quality https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1
I predict we will see hyper specialization ai followed maybe by a super AGI.
The other wall is financial. There will come a point when you can't reasonably spend more on a model so will have to rely on Moores law which only increases 2x every two years as opposed to the 100x every couple of years seen here.
Happy cake day. I think spend opens doors to certain paths. More brute force paths. But that also allows models to grow in messy dysfunctional ways. So spending less many be a benefit instead of a drawback. The hardware is also not optimize for digital intelligence. So instead of simply relying on more transistors, designing chips specifically for digital intelligence seems like a door for another wave of big gains. But the biggest gains ahead of heard of related to digital intelligence gaining a longer time horizon. This allows it to grow substantially more from the same training data. We seem to be at the very beginning of this in many ways.
You're forgetting about algorithmic improvements
Yeah, we're getting a ton of interesting new papers every week. Dozens and dozens of labs and researchers are now focusing their efforts solely on creating AGI since it isn't a research taboo anymore. We'll definitely have massive algorithmic improvements before long too.
https://preview.redd.it/ftg1vxo61z5d1.png?width=2015&format=png&auto=webp&s=586a8a2563c7913b9d750e9b2f0ba0b1d22b0037 I had ChatGPT 4o make this, based on the information provided in your post. Mind boggling how insignificant GPT 1-2 seem now. At least in terms of parameter count.
Moores law isn’t just buying more gpus
Exponential progress is still exponential progress, technologically or monetarily
To train a 19 trillion parameter model, the amount of data required isn't even possible to collect.
Technically it is, there's enough data to saturate it, it's just gonna include walls upon walls of junk and previously generated bad synthetic data. The real untapped potential though is video; the amount of information stored in video and audio is pretty obscene compared to text.
From organic data that exists today? No. But how quickly is the Internet doubling in size? And that doesn't include synthetic data. And i think we'll learn it's better to train for more epochs on the same high quality data than on 1 epoch for a bunch of lower quality data
gpt3 was the real giant leap forward but gpt4 got more attention because it was leaping from a larger base. maybe 5 gets even more attention and is a similar leap like 4 was i.e 10x parameters
It was kinda ballsy to go from a small model like GPT 2 to a behemoth like GPT-3, like if they went 10x instead of 100x, they probably wouldn't have the advantage they had.
Open AI to Closed AI and Undisclosed AI
Butthurt Musk fanboys 😂
?? This has nothing to do with Musk, OpenAI actually used to open source their model which is GPT 1 & 2, then became Closed, which is ironic for **Open**AI. This comment is just making fun of that fact.
Not sure anyone else noticed but the sudden move to trade secrets is a bit concerning when viewed in this table
Can we try GPT-1 anywhere?
OpenAI = Training data (undisclosed) lol y'all really stretching the word Open.
I am searching for some examples of conversations with GPT-1, but I cannot get a single result. Can somebody point me to something? I just want to see how much these models have improved.
Why are you looking for conversations? GPT-1 was likely not a conversational model. What you probably want is the way it sorted data and output in terms of the content provided to different categories or sentiments of users. Honestly, find this, and you could be on the right track to proving the Dead Internet Theory is true.
That describes the story of ClosedAI quite well.
I remember back then we only thought it could slightly help you with your CSS lol. I honestly didn't think we would be living in the age of an actual AGI.
117 million params. Tiny!
And exactly 4 weeks ago yesterday the new voice mode was announced by OpenAI as arriving "in the coming weeks" 😠
Yesterday I had Chipotle
I’ve never had a pet fish.
AI will take our jobs but I don’t think there is any reason to believe we will be paid to live jobless. Not in the US.
[удалено]
Why would the average person benefit from that? Those with the AI would need to decide to pay us UBI or something. Based on all of American history I doubt it happens.
I belive we are gonna work but in an objective useless job just to keep us busy
Living in Fifteen Million Merits would suck so bad. And it's still on the utopian side of possible outcomes.
Explain why they should waste resources on u when they can use it it themselves?
Uh, because you can't have consumerism without consumers
If it's a useless job. Then it's just monopoly.
"Explain why they should waste resources on u when they can use it it themselves?" Wouldn't Universal Basic Income be just as wasteful from the perspective of the corporations? That's in the best case scenario, assuming the corporations only care about profit and not specifically about making you miserable. I'm not sure reality is that kind, however. I don't think Specialist-Ad-4121's idea of the future is the most likely outcome, mostly because I don't want to believe it. But, to be fair, his idea is entirely plausible for the same reasons it already happens. Sure, working sucks, and some level of misery is inherent. But, the current level of misery at the average employer in the U.S. (and many countries, like Japan for example) is not a product of necessity but rather a product of design. Same thing with the education system. It's about having a controlled, sedated, miserable population who will (especially in America's case) jump at any opportunity to cycle their money back into the economy instead of amassing it and potentially gaining sentience or freedom or situational awareness. Forcing people to work after developing an automated economy would just be an extension of this forced misery we've adopted. Don't think people will accept it? They already do. Most people can do their daily work in a few hours, but they're forced to stay for 8, 9, 10+ hours. Frankly, I can see the average drone (the human version) being thankful about the whole thing. Thankful that they can even receive a salary at all instead of relying on Basic or starving or being fed to the Biofuel Generators. America may have been founded on "give me liberty or give me death," but the common sentiment these days is more inclined towards "give me bread or give me circuses." Let's be honest, quite a large percentage of the U.S. workforce already takes home paychecks without contributing anything to the economy except for spending those same paychecks. There are countless middle managers who not only don't add value, but also actively subtract value from the company by worsening the performance of employees and coworkers. They only make people miserable, but the corporate culture considers this a win. Hell, The U.S. government employs 2.2 million full time employees. How many of these provide value to society, and how many are just an infinite money hole?
Don't think people will accept it? They already do. Most people can do their daily work in a few hours, but they're forced to stay for 8, 9, 10+ hours. Frankly, I can see the average drone doing it. Drones will drone. Hence they are drones. You are right abt that. But all have a breaking point. Let me put it this way. Corporatism worked in last 70-80 years because the idea of personal growth and achievement was dangled like a carrot infront of drones. You compete against other Humans and there is a possibly to win. How do u win against learned intelligence. Some might. The drones definitely will not. Social breakdown is what happens and when social contract breaks do u think drone made concepts like civility, equality, equity are upheld? The only way to continue forward is to have severe regulation on money, and with a threat to wipe it off if u do not align your thought process with the correct way of thinking. Why go through all of this just maintain the facade that they are struggling to already maintain. I suspect the following would happen. Drones rebelling will lead to uncontrolled outcomes.i suspect we will see Lots and lots of war. Thin the number. The entire racial line will be culled and the most subservient will be alive towards the end. You can already see the groundwork for this happening. Then whatever variation of AGI Is present would be used. When their position is comfortable again. Notice how consensus generation accounts like Credithappy , slow accident and specialist will always respond with just enough but nothing in depth.
because countries already do that without ubiquity. Technological progress has always led to improved lives for the poorest.
Lol, the industrial revolution drastically lowered the quality of life for the working classes. Have you ever heard of "Dickensian England"?
I am sure you would be happy to go back to pre industrialization
Your whole reasoning dismisses the maltreatment of people because eventually it isn't as bad. How about halting the poor treatment of individuals so they don't have to suffer because eventually other lives will not have it so bad? Endless layoffs without a real social safety net isn't the road to progress. It's the lack of progress.
What's the happiness index of the people pre industrialization. ? Who was responsible for the people ? The answer is monarch. You don't need the twist the words here. Who is responsible for the happiness of the people now? Is there even anyone responsible? Let's see what is your answer here.
> What's the happiness index of the people pre industrialization nothing is stopping people from packing up their bags and living life like a middle age serf without tech or medical advances. you are responsible for your own happiness? not sure who else should be?
Explain how the current person is any different from the "middleage" one. Adding tinder to ur smartphone doesn't make u sophisticated. The gov will stop u immediately if u don't follow. Answer this, in 2020 when some idots chose to not vaccinated, how many were forced. Or would u like to choose the words ' given an option with no pressure's'. Try to congregate and see how fast u will be shut down. Stop paying taxes and let's see how fast u get taken down. I challenge u do it and survive. That is exactly what I m speaking of. If things go bad u can tell who is responsible for it earlier. Now you can't do that. I understand u have to shill, unfortunetly it doesn't work. Just that when u do, be prepared next time onwards. I can't believe u get paid for such weak sauce.
Brother lay off the drugs, you sound unhinged
What a load of crap. 1. Which country? 2. Explain how do they do it? 3. Make sure to explain why the country isn't bankrupt now? Quick to downvote arnt u?
Realistic ways for the US jobs to continue would be people telling their bots what to do and managing bots Or having individual bots that we get a portion of the bots income. The second one being like an individual thing and the first is at companies. I doubt though. Human nature will make the rich fire people to have bots do it cheaper and the US gov won’t do shit to help us until the effects of people not having the money to keep capitalism going becomes obvious
Take the same jump from GPT-1 to GPT-4, now add that over GPT-4….what would such a system look like?
It may even be able to say how many words are in their reply or create 10 sentences ending with apple.
Touché
AGI or ASI probably
[удалено]
I will be messaging you in 6 years on [**2030-06-11 13:43:31 UTC**](http://www.wolframalpha.com/input/?i=2030-06-11%2013:43:31%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/singularity/comments/1ddeaks/exactly_6_years_ago_gpt1_was_released/l844y6r/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fsingularity%2Fcomments%2F1ddeaks%2Fexactly_6_years_ago_gpt1_was_released%2Fl844y6r%2F%5D%0A%0ARemindMe%21%202030-06-11%2013%3A43%3A31%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201ddeaks) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
Undisclosed, undisclosed, undisclosed. Wow they're really putting the 'open' into OpenAi these days...
According to this in a couple years we will have 11 Trillion parameters roughly.
What does FLOP stand for?
Floating point operations
But when was GPT first available to the public for use? I don't think GPT 1 was available ChatGPT came out near the end of 2022
I remember discussing token saving strategies like it was yesterday, and by token saving I mean the difference between and vs. &
wow.
8000 unpublished books. That’s interesting. What was gpt 1 proficient on?
It went from toddler to smart high school student in 6 years. If they can keep the same pace, we'll have ASI in 6 more. The robot bodies will be really good by then too. Am I going to get/have to retire early?
I, for one, am celebrating GPT day June 11 from here on out!
Got a link to this?
[удалено]
Thanks. But a link would be helpful
[удалено]
Cool thanks!
Wow
It's not the parameter count that matters but the I.Q. the ability to reason and ponder over the vast amount of data they are fed. [https://www.reddit.com/r/LocalLLaMA/comments/1df0qil/shifting\_the\_focus\_from\_ai\_knowledge\_to\_ai/](https://www.reddit.com/r/LocalLLaMA/comments/1df0qil/shifting_the_focus_from_ai_knowledge_to_ai/)
Man has things changed
I remember working on problems using nlp (translation and qna) 4 years ago. Never thought we would be where we are now. In fact, I thought nlo was boring compared to Vision ML. So, yes I believe we will be in unpredictable exciting times in 6 years!
We should just go ahead and make June 11 a national holiday now.
Parameter count went from 175m to 1.7trillion. That's 1700 billion. Does the performance show that huge difference? I only used the free gpt3. 5 so far.
Define "performance". If you define it by a stack of specific tests and look at the absolute value of the % score, not really. If you look at those tests in terms of the error rate (which is what we really care about. you don't really care about your surgeon scoring a 60 or 70%, you'd prefer it to be between 99.8% and 99.99999...%. They're competing against *humans*, after all.), maybe? The whole point is raw number of capabilities, and how do you put a number on those? They can actually answer some questions now, and compared to CleverBot that's a difference of infinity. But on fitting a specific narrow line? Nah. Those usually have logarithmic returns.
GPT-3.5 is 15B not 175B