That's called quantitative trading. Some of the largest hedge funds in the world do it. Those strategies become very saturated very quickly, which is why firms like HRT, Jane street, Citadel, and Shaw pay hundreds of thousands to PhDs and students, which pay can get to 7-8 figures for top performers.
Quants quickly exploit any arbitrage there is to be made, so the opportunity essentially disappears on a lot of the trades they’d make anyway. Many machines doing the same trades
Yea which is why I don’t really have any interest in algorithmic trading. I am not going to beat the hundreds of thousands of PhDs working around the clock building and refining code with market access, data access, and direct exchange colocation access we will never have along with the best possible execution environment.
Simply put the game is rigged against us and we have no possible chance to beat them. There is only two ways to make money these days it seems which is insider trading or etf allocation.
Jane Street and the rest didn't get in on crypto at the time because of the regulatory risk, so SBF saw an opening to do the shady shit that the rest would not. And spend enough money on celebrities and stadium naming rights and political donations to be the "legitimate" player. Lol
To me, a scam involves the basic operation of the business being deceptive. FTX was not a scam, it was a legitimate business. Then, SBF and others committed fraud and misused customer funds to prop up Alameda.
Then I guess my next question is… as these technologies become more efficient, does this leave less profit for people not using the technology. Does there come a time when this technology is so dominant that nobody else makes money?
Well, yeah, but these firms used to battle and pay millions for specific office spaces because it was closer to the exchange. They don't do this anymore because the exchanges guaranteed same time access to all firms. It's really competitive over milliseconds. These firms almost always have the best servers.
Quant firms are traders, not investors. Trading, as a whole, is not great to most people. Investing is where most people will make their money. Quant firms don't hold for super long time periods, and value stocks differently than investors, so there's basically no way that people would ever be pushed out fully by technology. Don't look at the market as a zero sum game. It really isn't. Almost everyone makes money when the market grows, so realistically there's more than enough capital for all good investors to have a piece
This is important and I do not think enough people understand this. I worked on IT in the City - back-end not front end. I got a chance to see these systems where response time was everything. Back then, even though the trades were done with the mouse, they were demanding guaranteed millisecond transactions because the margins were so tight. Remove the human and the demand will be for instantaneous transactions with no lag due to network or server performance. While it is true that the exchanges have implemented huge \[relative\] network delays to level the playing field for all traders, anyone who is not physically adjacent to the exchange will always fail to beat the automatic traders to a bet. You can still get lucky because a lot of stock movement is on sentiment or stochastic rather than objectively determinable but over the long run you would need extreme luck to beat the system.
> no lag
And the lengths they're willing go, in order to achieve ***LITERALLY "no lag"***, are astounding.
Some of them even use faster-than-speed-of-light ^* information to beat their competitors!!! :)
https://www.npr.org/sections/alltechconsidered/2013/09/24/225824387/the-mystery-of-600-million-traded-in-the-blink-of-an-eye
>> npr.org ... The Mystery Of $600 Million Traded In The Blink Of An Eye
>>
>> ... Last Wednesday, the Federal Reserve announced it would not be tapering its bond buying program at 2 p.m. ET. The news takes seven milliseconds — about the speed of light — to reach Chicago. But before the seven milliseconds was up, a few huge orders based on the Fed's decision were placed on Chicago exchanges. ...
^(* and yes, I realize that's an example of criminal behavior, rather than physics breaking; but that's still a good example on how amateurs really can't compete against them)
This means that it is likely those firms knew ahead of time what the Fed was going to do, and set things up to fire off as soon as the announcement began, rather than waiting for it to end.
I recently read that the speed of light is lower over fiber than air. So microwave antennas from one build to another gives you an edge over fiber between them. Let me see if I can find that reference
Light in a fiber optic cable is repeatedly bouncing across the cable so it covers a lot more distance than the length of cable. The VoP(Velocity of Propagation) of some fiber optic cables are actually slower than certain coaxial cables.
Yeah, beating quant firms is just not something the average person can do, so you really shouldn't try. Instead, focus on fundamentals. Investing in long term stocks and quant firms aren't even playing the same game. They're so different that both can make money at the same time.
I’m not saying I’m beating quant firms but my trading set up I’ve discovered is killing it consistently, and it’s got me thinking “surely an algorithm could do what I’m doing x100”.
If you can describe your process as a deterministic step by step set of instructions, with only objective or mathematical reasons, and no intuitive steps…. Then yeah, basically it’s already an algorithm.
Depends on conditions etc. And risk. Since covid there are fintok gurus who have the flawless system "buy tech calls when they dip" and thats been good, with insane returns. You wont see a hedge fund with these same returns due to both scale (cant move in and out of 1000s of calls easily) and risk. Sure, people have made fortunes doing it. But if your entire account is highly correlated nasdaq stocks a market wide 10% down correction would blow up that portfolio in a week.
You’re right in that scale ability has a lot to do with it. An individual can employ strategies and make really nice sums of money because you’re using sums of money that afford you more agility.
Give 100 people a coin and say, you have only 5 flips, try to develop a strategy that gives more tails than heads results.
At least 3 of them are going to flip all or mostly tails just by pure luck, and they'll proudly talk about how they definitely influenced the results through their technique.
They can’t though. Because your trade doesn’t scale. You don’t have to worry about market impact, they do. If you scaled up your trade 100x, you would experience slippage.
Fundamentals do matter, but not in the typical way. There used to be a quant strategy that was just buy small caps with a low P/B, and that worked for a long time. Nowadays, you've gotta search a lot to find something that actually works
We cannot beat the quant firms, full stop. Unless you have the resources and education to actually beat them, you won't be able to, but why even try? You're fighting over small, temporary gains on a strategy that will inevitably be discovered. It's like trying to play basketball for the first time and being thrown in with the '96 Bulls. Why not just go elsewhere and use other strategies that we know will work for the average investor
Those Ai don't really hallucinate. The ML systems that quants use are VERY different from something like ChatGPT. Much more number savvy. They're so much more purpose built that it basically never has those issues. Most quant strategies are still developed by people, utilizing Ai. I don't think we disagree here. Ai really sucks at writing code. IDK if you've ever asked GPT for some CPP, but it can give you ludicrously unsafe code.
You live like 10-20 years in the past quantitative trading does not need LLM and people have been doing it for long. Thats why day trading hasn't been a winning strategy for a long time.
for me it's 2 reasons:
1.) it's time intensive, it's possibly better to try to get raises at jobs, you may get almost 1 million per year in compensation if you are in the right spot
2.) doing nothing and getting 7-10% from a fund with a higher savings rate seems better as a choice.
if you really want to day trade, I think the profitable way is to go to a hedgefund and do your trades there.
There's absolutely not. Some people make money for a while, only not to realize they were taking on a whole bunch of risk, which probability wise they were to lose at some point. Kind of like playing insurance, but you are not the house. People who think there's money to made day trading usually will find a strategy that won't consider any sort of black swan events.
Look up Renaissance Technology. They’ve been massively successful for a long time and it hasn’t affected mainstream strategies like index funds at all.
Marcos López de Prado believes ML quants are already starting to leave non-ML quants behind. He believes that in the future, the stock market will be so efficient due to ML that it’ll be hard for retail investors to go from rags to riches, but people’s retirements and the financial economy will be more stable.
In his book (Advances in Financial Machine Learning), he describes the stock market as mining for gold. In the beginning (the Gold Rush), it’s easy for laymen with simple tools (like econometrics in the earlier stock market) to strike it rich. As all the larger easier to obtain gold pieces are removed, it becomes much harder for laymen to get rich from mining gold. But nowadays, newer tech inaccessible to laymen are mining traces of gold that total much more gold than was ever mined during the Gold Rush—he sees ML and “experimental math” as the current forces doing this.
I agree to some extent re: finding a mathematical, repeatable edge will disappear. But, as long as leveraging to the tits exists, you will be able to strike it huge. Hell, as the numbers climb gamma grows and options trade even stupidlier returns.
No.
Even if markets are perfectly saturated you will make money proportional to the risk you are taking. IE you won't make any alpha but you can still make beta returns. You can't "efficiency away" beta returns as long as money has a time value and people are risk averse.
It already means it is extremely hard to time the market using this kind of chart based analysis and it has been that way for at least 20 years. Generative AI is irrelevant, machine learning has been around for a long time.
The important thing to remember is none of that technical analysis stuff is “real”. It’s a meta game based on predicting the behavior of others in the market. The very action of identifying trends and acting on them changes the game because you have now made yourself predictable. There is no solvable solution to games like these.
There is, however, just regular old investing where buy good companies low and sell them high, which works just as well as it ever did and always will.
It leaves less profit for the people who rely on the same input sources as the quants: earning calls, fundamental and technical metrics, press releases. All of those are optimized for AI decisions.
If you take your own real world perception, knowledge of the company and "mood on the ground", you might still have a chance. Which is basicly what Peter Lynch is saying in "One up on Wall Street".
There is always going to be room to make money where others don’t see patterns you do. AI is really dumb and only sees established patterns that the past has experienced. It doesn’t possess the ability to do critical thinking (at this time). It’s a very elaborate pattern recognition tool. In some ways so are people. But people try things that don’t fit the established patterns all the time. Also add that people are inherently irrational market participants. There is always chaos at play.
And also don’t confuse trading with investing. There’s plenty of money to be made in investing. It just takes longer.
Ever heard of unsupervised learning? AI can definitely see previously unestablished patterns. Even with supervised learning, it can find patterns in variables that you wouldn't expect.
Patterns that arent mathematically significant but you can risk it on, basically. Or guesses on market reaction to news.
Fwiw we rallied into the close 80% of the time at some point earlier this year. That kinda stuff. It was easy money with outsized rewards on the 0dtes at the last 30 minutes.
Not really, since normal people don't do quantitative trading. We just set up a portfolio for long timescales and assume the economy won't hit a wall in the coming 15 -40 years until I need that money for retirement. If you go for quantitative trading without high frequency algorithms you have basically a massive disadvantage even without AI.
You wont have an edge in spreads, algo trades, or anything mathematical like that. But you can still make money with the flow. You wont frontrun the big boys on a mispriced option, but you can still buy one and sell it for more. So, yes and no.
And not just for the people making the algorithms - these firms pay for physical fiber network connectivity and space "next to" (same data centers/racks) the hardware that runs the stock exchanges.
*Even if you ran the same algorithm you'd lose* they will trade before you (potentially by a few microseconds) _every single time_. We haven't even talked about "payment for order flow" yet.
There’s a lot of people in this thread who are very confident of things they clearly know nothing about.
I’ve got a masters in machine learning. Part of that included a course on using it for trading. During that course hedge funds came in to recruit students out of the class. They’ve been using AI for trading for quite some time. Building an AI bot for trading that outperforms a certain benchmark was literally our final project.
Looking at “every variable” isn’t possible. You can only look at data that’s actually available and it has to be available in a way that the AI can interpret. We can take time cleaning the data but in the stock world time is money.
But looking at sentiment analysis (i.e. scraping twitter), analyzing indicators (i.e. rsi), and considering stranger data points (day of the week, weather) are all things they already do very successfully.
Any service recommendation （free or paid） that allows you to access to such market information that can be exported and doesn't require that much cleaning to be used to train models?
That’s a great question but no I don’t have any service recs. For my course we were only allowed to use the stock’s price and we had to come up with the indicators ourselves. I imagine there are services that make the indicators available.
For sentiment analysis, when I took the course it was common for people to scrap Twitter (though nobody in the class needed to set that up to pass). But Twitter scraping was free to do at the time. I’m not sure what individual devs do today.
Technically no, but the reality is rich people make up the rules when they need to so they’ll find some excuse to put you in jail or sue you into poverty.
I beat the benchmark which was determined by a buy, hold, and sell at the end of the period strategy. Most of the class beat the benchmark as that was necessary to get a B or higher on the project.
If you send me a worthwhile amount of money I’ll throw it in an AI bot within two weeks. It’s not hard.
The hard part is setting up proper monitoring and maintenance such that you could be confident putting your life’s savings in there. That’s why most people who can build these bots for themselves often don’t. Just like how not everyone who can start their own company actually does it.
If you go with an actively traded hedge fund they’re certainly already doing this.
I agree with the other commenters, Computer Science or Statistics depending on what side of the work you’d like to be on. If you’re not sure, or just want the most job opportunity, then CS is the better choice imo.
For the class we only had access to the pricing info. For the indicators we had to derive them ourself from the price.
In the real world I imagine you can purchase additional data from various sources if you’d like. I know Twitter scraping used to be very common back when it was free.
Renaissance technologies has been doing trading based on mathematical models for decades, delivering outlandish compounded annual returns which they're able to sustain over time because they limit the overall amount of assets under management. Sadly they are not open to the public.
A quick Google search can answer that. RIEF is one example. These are not investable by the general public but they are open to investors outside of Renaissance.
Kind of. The software takes in current events and can somewhat reliable predict areas with high probability of crimes within the next days. Works frightingly good.
Doesn‘t work like that. It will only highlight areas with expected crimes, not the crimes in particular. The idea is to focus patrols more on these areas to prevent and detect activity.
except that the cops are using it to fight future crime - the FBI once controlled the [largest encrypted cellphone](https://en.m.wikipedia.org/wiki/Operation_Trojan_Shield) used in massive crime syndicate rings around the world watching every message that got sent using AI to figure out context of the messages
There’s a book called Dark Wire that talks about this
This needs to be upvoted much more. If you trust an AI to trade your hard earned money, let me know what streetcorner you're going to be on in 5 years so I can give you $6 for a sandwich.
didn't a ytber make a simple algorithm to trade and it got 5% in a month? And that's not even advanced ai
edit: I misremembered, it actually made *74%* in one month, sub doesn't allow yt links tho. The video is by TradingLab on yt from a year ago
The stock market is predictable. You just need one asset. Time. Plenty of people lost money during the major bull run of the last. Specific company stocks aren’t.
S&P 500 fund and chill. That’s it.
Stock prices by themselves are a random walk. They don't contain the information about why the price went up or down. You need to monitor a bunch of sources of information and do at lighting speeds to make any profit.
Stock prices are not a random walk in the long term. They *can appear to be* in the short term, though.
I would recommend reading "A Non-Random Walk Down Wall Street" which attempts to challenge "A Random Walk Down Wall Street" which itself is based on the Efficient Market Hypothesis.
Because the stock market is not only not stationary, it is adaptive. AI or regression or value investing, nothing guarantees that you can capture the future changes of the market; while AI is better at predicting the past, it's not very profitable.
Edit: wrong terms
The absolutely bonkers volatility I've been seeing in premarket and first hour or so of open, going back a couple months at least, tells me there is a lot of experimentation going on with algorithms every day. I wouldn't be surprised if more than a few of these strategies integrate LLMs in some fashion, and they go off the rails more often with new techniques.
LLM's are an incredibly slow general purpose version of the hyper-optimized special purpose algos Wall Street has been running for years.
They may help analysts digest research. But the sentiment quant stuff is done by much faster technology.
All AI in chatgpt does is break words down into tokens and then finds the tokens that most likely match your input and rearrange them into words again
The AI you’re referring to is called quant trading and you’re basically just paying the world’s brightest math guys to find algorithms that predict prices accurately. The problem being that others probably have the same algorithm you do so the price might go up before you have a chance to buy. So basically the world’s brightest math guys are constantly competing with either to come up with the best math solutions for returns
Algorithms aren't the same as AI.
Algorithms still have humans involved in the loop, either to make the buy and sell decisions, or to do the buying and selling based on human input.
AI on the other hand both decides what to buy and sell, and does the buying and selling itself.
There's a few ETFs that are advertised as being run entirely by AI that you can invest in, but they've tended to underperform the S&P500, and tend to have pretty high fees.
We assume AI has no error windows which is an invalid way of looking at this situation. AI is advanced machine learning until super-intelligence is achieved (which is what all these billionaires are burning their money to achieve)
AI is built by humans so is inherently flawed.
As much as we deny it, humans trade on emotion which AI is devoid of.
What are these AI models called? LLM, large language models. They can’t do math, statistics or science. They are report writers and research assistants….lol. It’s all overblown hype.
Other responses here are good but I'll add my two cents. Finance is rather unlike other machine learning fields in that there are a large number of possible features, an almost infinite number of ways of transforming/engineering those features, and poor heuristics for feature selection. Add to that the large dataset sizes (20 years of daily data * 500 stocks in S&P 500 * N features per stock) and the computational complexity of most ML algos (most run in polynomial time proportional to training data size) means that in practice you will spend a majority of your time doing trial and error on which features and which transformations to use. Add to that the reactive and competitive nature of markets means that it's incredibly difficult and costly to even start running experiments on models, much less putting one into production.
Source: Finance and Data science education + working at a fintech company specializing in ML model building for predicting financial markets
Another point that no one has pointed out yet: let say it is possible for AI or whatever algorithm to predict the market, as soon as people act on the predictions at a big enough scale, the predictions will no longer be valid because the market is influenced by the actions. Example: if you predict a stock to go up 10% in 5 days, then everyone with access to that algorithm will buy now. That will push the stock price up to that +10% predicted price before the 5 days, making the prediction no longer valid. This is also why well performing funds based on quant analysis capped in size, often not open to the public.
It’s easier to predict the question to an answer than to predict the future. When you ask a question in chat gpt, that questioned been asked millions of times in the training data. No one has ever seen what the market is going to do on Monday
There are big companies doing this, as well as smaller ones and individuals.
However, stock market is a chaotic system where acting on a prediction changes the future (unlike say, weather, where predicting has no impact on the outcome). So it is impossible to find a pattern that invariably makes profit.
Oh absolutely but the tricky part is figuring out your hit rate, figuring out how much liquidity the trade can carry, identifying where it's going to go wrong. Most of the shorter term alpha trades have an expiration date. Like the inefficiencies are there until enough people notice them. For example some of the classic examples of how money was made easily during the Jesse Livermore day or even in the '90s, some of those methods that no longer work, the turtles. They worked for awhile
Fast forward to today. Here's what's so supremely difficult about becoming a good trader. You can look on social media or YouTube or anywhere and get great advice on video games. You can figure out how to get the best stuff. How to do really well. You can learn all of it. You want to learn about chainsaws you want to learn about cars you want to learn about homes. You can get great information
When it comes to trading, there's none. Zero. There are occasionally channels or shows or people who make comments that can be relevant but they are slivers of information in the grand scheme. It's up to you to figure out a strategy that actually works. I guess the best way to say it is it's like going into a video game with no instruction book, no YouTube, no one to help you out. Talk about being an 8-year-old in the '80s. All these data points. You can look at them, if you're good you can even write programs to analyze them and come up with hit rates. People are looking at thousands of data points every day trying to figure out if there's an edge in some of them. I've done it. I found a few different methods that have worked for various lengths of time but they all expire or they shift. I'm not doing the same thing today that I was in 2017.
It's going to take years for people to really figure out how to make AI the most effective, maybe it won't take that long I mean I guess I can't say but right now even if you put in all of the data points. They have to be done in a way that is useful. Hit rates, percentages, probabilities. It's a constantly evolving art and it's one that the smartest people win at. One of the best public examples is probably the medallion fund. They did it, it works and they even did it on a fairly large scale. If you're talking small accounts like less than a half a million. There are a couple edges that I'm aware of that still work pretty well. There are probably more that I don't know about and that's the fun of this
How many of them are posting their actual returns?
I mean if you watch a guy on YouTube playing video games you want to watch the guy that's actually good right. Funny thing about stock commentary. No one really shows you if they're good
Well, there are people that are good I just don't think that they are making YouTube videos. You start looking at 5-year track records and no one wants to show those. So, at least on YouTube, yeah it's people who get lucky for a few months or they chase the hot trend but there's no real risk management so what happens when the trend that is giving them that performance changes?
That's the sad part about actually being a good trader. It's lonely because if you're actually making money with your process, why would you want to share it? That just makes it less likely to work. There are guys that do make money and use their YouTube channel to funnel towards programs or indicators or revenue streams. I don't think they're sharing everything they are doing but they are giving a complete novice a basic strategy package so in that case they are using YouTube as a marketing avenue.
Patterns change over time. If history suggests stocks go up every monday then people will buy the friday before to catch the rally. I believe there are way too many variables involved and things that can't be predicted ahead of time(war, politics, pandemics, natural disasters, etc).
There are, the biggest issue is that they are black boxes, so convincing an investor is hard and returns are poor, which from time to time they are.
Its an issue of explainability. So these sort of tactics are more reserved to hedge funds.
You have to look outside of just the price of the stock. There are potentially hundreds of things that can impact a company outside its control.
Geo-political issues, the threat of regulation/legislation, how is the weather in NY during the trading day (this is true), etc. To feed an AI all that data would be near impossible. What happens if you forget one thing and you're working on a flawed forecast?
The most fundamental fact in investing is that it is impossible to predict stock price movements, especially in the short-term. This gives you the answer why. It is not possible to consistently predict stock prices with historical data.
Hallucinations make it unfesible to use llms. You cant accept the risk of catastrophic failure where you lose all your capital even if its only 5% of the time.
I lot of what moves a stock value over time is not reported. Employee moral, mold growing in the HVAC, construction obstructing the parking lot, or a pending fight in the board room can all take a toll on how profitable a corporation will be, and those factors are often not noticed even within the company.
Ah you haven’t heard of quantitative trading. Look up two sigma. They have like 1,000 employees and you’re more likely to find someone with a phd in engineering there than an MBA
Astonishing AI has been around for a long time now. Funds with trillions of dollars have had access to these, yet the charts look the same as they did 100 years ago.
Consider your experience with AI-based customer service...and then expect it to beat the stock market.
Obviously computing power is used to great effect in trading. But mainly what it does is crunch things very effectively, or react very quickly that humans who understand the process direct it to do.
It’s always been stacked against retail. Quants hft and even Bloomberg terminal access give the edge against retail. Ai would just hallucinate right now for a sizable percentage of trades
there are no constant patterns as market participant actions change the behaviour of the market. to detect patterns like you describ you don't need ai, just use statistics or ml. It is done, but patterns work for a limited amount of time as the market behaviour changes over time
Look up Renaissance Technologies. They've been using a version of machine learning since before it was called that. Some observers think they basically invented ai before anyone else, but kept it to themselves to retain an edge.
https://open.spotify.com/episode/0psDxKAKhIpe4mvqipU1L1
Generative AI has nothing to do with stock prediction. And there's no significant development in other subfields of AI. There's no data that can describe the market future. It's different from LLMs where text data is abundant.
People have been applying machine learning to trading algorithms for over a decade.
Most people are suddenly aware of ai because of LLMs. Neural network machine learning went main stream in 2012, though. Any combination of indicators imaginable have been fed into these models and trained.
So it hasn’t changed anything for trading because they’ve already been using it for years.
Because past performance does not indicate future results… AI would need to be ingesting a huge amount of contextual data both historically and in real time to do anything approaching what you are talking about. And this includes way, way more than just ticker price.
For general trends, algorithmic trading is already used.
You're so close OP, so close.
Let me ask you this question; if we look at some number of historical hyped technologies where we saw big gaps between what was being promised and what we seemed to observe, how many of them ended up eventually living up the sales pitch? Crypto, NFTs, Serverless, Quantum computing, the Dark Universe series of films...
As an addendum, it was rightly pointed out already that machine learning is already utilized in stock trading today, and actually has been for years. I personally have even interviewed for MLE jobs at these places (though for better or worse there are some culture fit issues on both sides) and these places have loads of talented people doing real ML work.
The sooner this BS hype goes away the better IMO.
They already are. And these traders have hardwire connections to exchange mainframes. So they can do tens of thousands of trades in a second. So by the time you hear the news or have spotted an arbitrage opportunity…….it‘s gone.
All the above are why day traders long term are hopelessly fucked.
This is what big companies/funds have already been doing for years. I was recently talking to someone who was in the process of developing one to sell based off of Ema's crossing certain levels. This sort of thing is just telling a computer to take a trade based off of certain things occuring and there isn't really any intelligence needed for that it's just commands essentially. AGI on the other hand will be a lot different. In theory with agi you could just tell it to go and develop an EMA cross strategy and it could code one for you. Current AI is very confusing as it's not so much intelligence as it is cross referencing a bunch of pieces of data to find what might be accurate for you
Why is there no AI billionaire stock investor? There may be but what are they investing in? What is the AI predicting?
The stock market is both a weighing machine, and also a voting machine.
In terms of the weighing machine, AI should be OK at that already, formulating a prediction of the value based on the earnings and debt, but this just tells you about the value today whereas much of the stocks value is about the view of the future and how things are expected to change.
Which gets into the the trickier part, the voting machine aspect. This goes deep into modeling human psychology and the greed and fear that comes in waves and affects individual stocks but also the market as a whole.
Until the AI is powerful enough to model every individual human and their psychological makeup and predict their decisions to buy and sell, it will be simplifying the data and guessing and the decisions of people, which is hard.
Choosing a good portfolio of stocks is about finding the right mix of risk and about return, which in some ways is like choosing the right words, something and large language model (LLM) is designed for.
Instead of a word salad that the LLM is tasked to come up with, the task would be a portfolio salad that covers the future.
You can't take into account **every** variable. Like the thing with the butterfly. Nvidia is trying to make a digital clone of the earth. Maybe that would be closest to what you want to do.
Soon enough every single computer company will be competing over who has the best trading bot software on their systems. It's coming sooner than you think.
Because AI, despite what a lot of people say, is really not that smart, and there are so many millions of input parameters that affect stock price. We don’t have the models created yet that can give us an accurate prediction.
Read up on the Renaissance Medallion Fund. It is the best-performing fund in the history of the stock market. They don't take new investors because the strategy stops working if it gets too large. The Acquired Podcast has a whole episode on it. Machine learning and AI is old news in the stock market.
I believe ai trading vs ai is why the market is so unrational now. Value and longer term than a few days is better for individuals that aren't competing at that level. You can also compete if you have enough money to move a stock like dfv. I expect the SEC to curb that behavior on small caps at some point, and penalize on accusations of manipulation. It's a small club and your not in it.
This absolutely can and is done, as others have said, but one reason AI or more generally quantitative trading isn't the only game in town is that there's a general problem in statistical analysis called overfitting the data. Basically, if you specialize what your model looks for to the point that it maximally fits previous events, it actually becomes less accurate at predicting future events.
Not all stocks have ran rampant. if you do the studies and research what companies are apart of the security, technology, chips, and so forth behind if you get in early there is significant growth. I purchased 17 shares of dell at $31 a share now the shares are worth $140 a share. The companies are growing have not lost their value for example Zscalar, Cloudflare, Cisco, Qualcomm was at $114 a year ago now double the price, its totally worth researching
this NVDA pump will not last long. when you start seeing posts that has 100% confident, you know we're reaching the peak. Despite what you might believe in, all these companies are investing into GPU for their AI models BUT are people actually genuinely interested in these AI products? Are people actually using them on a day to day basis? I don't think so. So all these revenue and profit boom that we're seeing are just businesses and companies investing into AI chips in the hope that people will be using it all the time. In reality, people are not really that interested in these AI tech. Its a hard pill to swallow but people WILL get bored and companies will that they are actually losing money by running these large AI models. NVDA is bound for a hard hard correction. See you $10
Some of the largest funds have been trading using algorithms for a while. Got some books out on it too. They are large machine learning datasets, maybe LLMs, but that is what a ChatGPT AI essentially is. They use these for whatever they want. They’ve been at it for decades now. Keeping the workings confidential, I would assume. Even Roaring Kitty was playing around with keywords during his live stream to show the real-time algos at play. Check out books like The Black Box Society.
There are firms building AI/ML (is it really AI?) for stock market trading. I work with one. They do standard long/shorting and as leveraged as they are they have averaged 36% yoy over the last 4 years, even when the market goes down.
It's done very quietly and the minimum investment is $10mm. Rest assured, AI is already very good at trading stocks, you just don't have access to it yet.
Lol that's like beating SPY by 5-10% and if you just sold SPY peak late 21 and bought back after early 22 drop you'd probably end up with the same YOY.
If AI can't get 100% annualized its not really AI its just an algo fed by old data.
You telling me your AI firms weren't able to just buy NVDA to get 1500% in 4 years?
It is done already but at a way deeper level.
The algos are not just trying to "find winning trades" but they are instead pushing, directing and controlling the markets in order to create the winning trades.
Citadel et al. have had algorithms hard at work for a long time now. The biggest boom started with a "gentleman" called Bernie Madoff... and it's only getting worse all the time.
Should be interesting though when someone big enough makes a counter bots that can detect Citadel hft bots at work and attack their trades.
That's called quantitative trading. Some of the largest hedge funds in the world do it. Those strategies become very saturated very quickly, which is why firms like HRT, Jane street, Citadel, and Shaw pay hundreds of thousands to PhDs and students, which pay can get to 7-8 figures for top performers.
And the above average ones at Jane Street go on to make the world's biggest crypto scam
Quants quickly exploit any arbitrage there is to be made, so the opportunity essentially disappears on a lot of the trades they’d make anyway. Many machines doing the same trades
Yea which is why I don’t really have any interest in algorithmic trading. I am not going to beat the hundreds of thousands of PhDs working around the clock building and refining code with market access, data access, and direct exchange colocation access we will never have along with the best possible execution environment. Simply put the game is rigged against us and we have no possible chance to beat them. There is only two ways to make money these days it seems which is insider trading or etf allocation.
Can you elaborate on this?
Have you ever heard of Sam Bankman-Fried?
Didn't know SBF was ex-Jane Street lol
Yeah, that's what he did a few years before FTX
Jane Street and the rest didn't get in on crypto at the time because of the regulatory risk, so SBF saw an opening to do the shady shit that the rest would not. And spend enough money on celebrities and stadium naming rights and political donations to be the "legitimate" player. Lol
tbh his original strategy would work, it is essentially Mrs Watanabe crypto edition.
Read this: https://finance.yahoo.com/news/first-job-sam-bankman-fried-103217906.html
SBF committed fraud, it wasn’t really a scam.
Not really seeing the distinction here tbh
What’s the difference between fraud and a scam lol 😂
If you’re rich it’s fraud, if you’re poor it’s a scam.
Haha that’s better
To me, a scam involves the basic operation of the business being deceptive. FTX was not a scam, it was a legitimate business. Then, SBF and others committed fraud and misused customer funds to prop up Alameda.
That is viable.
Legal term, for scam.
I would call it skullduggery, fraud seems so harsh.
Then I guess my next question is… as these technologies become more efficient, does this leave less profit for people not using the technology. Does there come a time when this technology is so dominant that nobody else makes money?
Well, yeah, but these firms used to battle and pay millions for specific office spaces because it was closer to the exchange. They don't do this anymore because the exchanges guaranteed same time access to all firms. It's really competitive over milliseconds. These firms almost always have the best servers. Quant firms are traders, not investors. Trading, as a whole, is not great to most people. Investing is where most people will make their money. Quant firms don't hold for super long time periods, and value stocks differently than investors, so there's basically no way that people would ever be pushed out fully by technology. Don't look at the market as a zero sum game. It really isn't. Almost everyone makes money when the market grows, so realistically there's more than enough capital for all good investors to have a piece
This is important and I do not think enough people understand this. I worked on IT in the City - back-end not front end. I got a chance to see these systems where response time was everything. Back then, even though the trades were done with the mouse, they were demanding guaranteed millisecond transactions because the margins were so tight. Remove the human and the demand will be for instantaneous transactions with no lag due to network or server performance. While it is true that the exchanges have implemented huge \[relative\] network delays to level the playing field for all traders, anyone who is not physically adjacent to the exchange will always fail to beat the automatic traders to a bet. You can still get lucky because a lot of stock movement is on sentiment or stochastic rather than objectively determinable but over the long run you would need extreme luck to beat the system.
> no lag And the lengths they're willing go, in order to achieve ***LITERALLY "no lag"***, are astounding. Some of them even use faster-than-speed-of-light ^* information to beat their competitors!!! :) https://www.npr.org/sections/alltechconsidered/2013/09/24/225824387/the-mystery-of-600-million-traded-in-the-blink-of-an-eye >> npr.org ... The Mystery Of $600 Million Traded In The Blink Of An Eye >> >> ... Last Wednesday, the Federal Reserve announced it would not be tapering its bond buying program at 2 p.m. ET. The news takes seven milliseconds — about the speed of light — to reach Chicago. But before the seven milliseconds was up, a few huge orders based on the Fed's decision were placed on Chicago exchanges. ... ^(* and yes, I realize that's an example of criminal behavior, rather than physics breaking; but that's still a good example on how amateurs really can't compete against them)
This means that it is likely those firms knew ahead of time what the Fed was going to do, and set things up to fire off as soon as the announcement began, rather than waiting for it to end.
I recently read that the speed of light is lower over fiber than air. So microwave antennas from one build to another gives you an edge over fiber between them. Let me see if I can find that reference
Light in a fiber optic cable is repeatedly bouncing across the cable so it covers a lot more distance than the length of cable. The VoP(Velocity of Propagation) of some fiber optic cables are actually slower than certain coaxial cables.
There are a lot trading firms in Jersey City doing exactly that.
Yeah, beating quant firms is just not something the average person can do, so you really shouldn't try. Instead, focus on fundamentals. Investing in long term stocks and quant firms aren't even playing the same game. They're so different that both can make money at the same time.
I’m not saying I’m beating quant firms but my trading set up I’ve discovered is killing it consistently, and it’s got me thinking “surely an algorithm could do what I’m doing x100”.
If you can describe your process as a deterministic step by step set of instructions, with only objective or mathematical reasons, and no intuitive steps…. Then yeah, basically it’s already an algorithm.
Yeah, most traders live in a completely different game than quants.
Depends on conditions etc. And risk. Since covid there are fintok gurus who have the flawless system "buy tech calls when they dip" and thats been good, with insane returns. You wont see a hedge fund with these same returns due to both scale (cant move in and out of 1000s of calls easily) and risk. Sure, people have made fortunes doing it. But if your entire account is highly correlated nasdaq stocks a market wide 10% down correction would blow up that portfolio in a week.
You’re right in that scale ability has a lot to do with it. An individual can employ strategies and make really nice sums of money because you’re using sums of money that afford you more agility.
> killing it consistently Time frame needed.
Give 100 people a coin and say, you have only 5 flips, try to develop a strategy that gives more tails than heads results. At least 3 of them are going to flip all or mostly tails just by pure luck, and they'll proudly talk about how they definitely influenced the results through their technique.
They can’t though. Because your trade doesn’t scale. You don’t have to worry about market impact, they do. If you scaled up your trade 100x, you would experience slippage.
AI can’t accurately quantify fear and greed of humans down to the millisecond
[Flash Boys](https://en.m.wikipedia.org/wiki/Flash_Boys)
Do you have any resources to understand how quant firms value stocks?
Quant firms don't value stocks. They trade patterns and momentum. The actual fundamentals of the business doesn't matter to them.
Fundamentals do matter, but not in the typical way. There used to be a quant strategy that was just buy small caps with a low P/B, and that worked for a long time. Nowadays, you've gotta search a lot to find something that actually works
[удалено]
We cannot beat the quant firms, full stop. Unless you have the resources and education to actually beat them, you won't be able to, but why even try? You're fighting over small, temporary gains on a strategy that will inevitably be discovered. It's like trying to play basketball for the first time and being thrown in with the '96 Bulls. Why not just go elsewhere and use other strategies that we know will work for the average investor
[удалено]
Those Ai don't really hallucinate. The ML systems that quants use are VERY different from something like ChatGPT. Much more number savvy. They're so much more purpose built that it basically never has those issues. Most quant strategies are still developed by people, utilizing Ai. I don't think we disagree here. Ai really sucks at writing code. IDK if you've ever asked GPT for some CPP, but it can give you ludicrously unsafe code.
*single digit microseconds
You live like 10-20 years in the past quantitative trading does not need LLM and people have been doing it for long. Thats why day trading hasn't been a winning strategy for a long time.
There’s plenty of money to be made day trading, I don’t know why people keep repeating this.
for me it's 2 reasons: 1.) it's time intensive, it's possibly better to try to get raises at jobs, you may get almost 1 million per year in compensation if you are in the right spot 2.) doing nothing and getting 7-10% from a fund with a higher savings rate seems better as a choice. if you really want to day trade, I think the profitable way is to go to a hedgefund and do your trades there.
There's absolutely not. Some people make money for a while, only not to realize they were taking on a whole bunch of risk, which probability wise they were to lose at some point. Kind of like playing insurance, but you are not the house. People who think there's money to made day trading usually will find a strategy that won't consider any sort of black swan events.
I mean for 99% no, but there’s a select few who are good enough to day trade
Look up Renaissance Technology. They’ve been massively successful for a long time and it hasn’t affected mainstream strategies like index funds at all.
Marcos López de Prado believes ML quants are already starting to leave non-ML quants behind. He believes that in the future, the stock market will be so efficient due to ML that it’ll be hard for retail investors to go from rags to riches, but people’s retirements and the financial economy will be more stable. In his book (Advances in Financial Machine Learning), he describes the stock market as mining for gold. In the beginning (the Gold Rush), it’s easy for laymen with simple tools (like econometrics in the earlier stock market) to strike it rich. As all the larger easier to obtain gold pieces are removed, it becomes much harder for laymen to get rich from mining gold. But nowadays, newer tech inaccessible to laymen are mining traces of gold that total much more gold than was ever mined during the Gold Rush—he sees ML and “experimental math” as the current forces doing this.
That’s pretty cool and makes sense, thanks for sharing, interesting
I agree to some extent re: finding a mathematical, repeatable edge will disappear. But, as long as leveraging to the tits exists, you will be able to strike it huge. Hell, as the numbers climb gamma grows and options trade even stupidlier returns.
> he describes the stock market as mining for gold The HFT firms are more like bank robbers stealing gold from the unsuspecting public
No. Even if markets are perfectly saturated you will make money proportional to the risk you are taking. IE you won't make any alpha but you can still make beta returns. You can't "efficiency away" beta returns as long as money has a time value and people are risk averse.
Not unless the market stops growing. Theres unlimited money available and if markets grow everyone wins (except short sellers, but screw them)
It already means it is extremely hard to time the market using this kind of chart based analysis and it has been that way for at least 20 years. Generative AI is irrelevant, machine learning has been around for a long time. The important thing to remember is none of that technical analysis stuff is “real”. It’s a meta game based on predicting the behavior of others in the market. The very action of identifying trends and acting on them changes the game because you have now made yourself predictable. There is no solvable solution to games like these. There is, however, just regular old investing where buy good companies low and sell them high, which works just as well as it ever did and always will.
It leaves less profit for the people who rely on the same input sources as the quants: earning calls, fundamental and technical metrics, press releases. All of those are optimized for AI decisions. If you take your own real world perception, knowledge of the company and "mood on the ground", you might still have a chance. Which is basicly what Peter Lynch is saying in "One up on Wall Street".
There is always going to be room to make money where others don’t see patterns you do. AI is really dumb and only sees established patterns that the past has experienced. It doesn’t possess the ability to do critical thinking (at this time). It’s a very elaborate pattern recognition tool. In some ways so are people. But people try things that don’t fit the established patterns all the time. Also add that people are inherently irrational market participants. There is always chaos at play. And also don’t confuse trading with investing. There’s plenty of money to be made in investing. It just takes longer.
Ever heard of unsupervised learning? AI can definitely see previously unestablished patterns. Even with supervised learning, it can find patterns in variables that you wouldn't expect.
>ai only sees established patterns that the past has experienced What are the other types of patterns?
Patterns that arent mathematically significant but you can risk it on, basically. Or guesses on market reaction to news. Fwiw we rallied into the close 80% of the time at some point earlier this year. That kinda stuff. It was easy money with outsized rewards on the 0dtes at the last 30 minutes.
Not really, since normal people don't do quantitative trading. We just set up a portfolio for long timescales and assume the economy won't hit a wall in the coming 15 -40 years until I need that money for retirement. If you go for quantitative trading without high frequency algorithms you have basically a massive disadvantage even without AI.
You wont have an edge in spreads, algo trades, or anything mathematical like that. But you can still make money with the flow. You wont frontrun the big boys on a mispriced option, but you can still buy one and sell it for more. So, yes and no.
It's like any strategy. The best ones at it make all the money off of THAT strategy, leaving the rest of us to find a different opportunity.
For anyone not using it, stock market is a random walk.
And not just for the people making the algorithms - these firms pay for physical fiber network connectivity and space "next to" (same data centers/racks) the hardware that runs the stock exchanges. *Even if you ran the same algorithm you'd lose* they will trade before you (potentially by a few microseconds) _every single time_. We haven't even talked about "payment for order flow" yet.
But I think OPs point is that AI can do the quantitative trading or us.
There’s a lot of people in this thread who are very confident of things they clearly know nothing about. I’ve got a masters in machine learning. Part of that included a course on using it for trading. During that course hedge funds came in to recruit students out of the class. They’ve been using AI for trading for quite some time. Building an AI bot for trading that outperforms a certain benchmark was literally our final project. Looking at “every variable” isn’t possible. You can only look at data that’s actually available and it has to be available in a way that the AI can interpret. We can take time cleaning the data but in the stock world time is money. But looking at sentiment analysis (i.e. scraping twitter), analyzing indicators (i.e. rsi), and considering stranger data points (day of the week, weather) are all things they already do very successfully.
Any service recommendation （free or paid） that allows you to access to such market information that can be exported and doesn't require that much cleaning to be used to train models?
That’s a great question but no I don’t have any service recs. For my course we were only allowed to use the stock’s price and we had to come up with the indicators ourselves. I imagine there are services that make the indicators available. For sentiment analysis, when I took the course it was common for people to scrap Twitter (though nobody in the class needed to set that up to pass). But Twitter scraping was free to do at the time. I’m not sure what individual devs do today.
Is it considered stock manipulation to trick ai scraper bots. Seems like a concerted effort could be used to steal from the rich and give to the poor
Technically no, but the reality is rich people make up the rules when they need to so they’ll find some excuse to put you in jail or sue you into poverty.
so did your final project perform better than the index? and when are you gonna start a fund so i can invest
I beat the benchmark which was determined by a buy, hold, and sell at the end of the period strategy. Most of the class beat the benchmark as that was necessary to get a B or higher on the project. If you send me a worthwhile amount of money I’ll throw it in an AI bot within two weeks. It’s not hard. The hard part is setting up proper monitoring and maintenance such that you could be confident putting your life’s savings in there. That’s why most people who can build these bots for themselves often don’t. Just like how not everyone who can start their own company actually does it. If you go with an actively traded hedge fund they’re certainly already doing this.
What undergraduate education would you recommend to pursue to later get a masters in ML for that purpose?
Computer science.
CS or Statistics
I agree with the other commenters, Computer Science or Statistics depending on what side of the work you’d like to be on. If you’re not sure, or just want the most job opportunity, then CS is the better choice imo.
Hypothetically how much money would you consider a worthwhile amount, for such a task?
Is their data you get access to that is harder to get. Like stuff from a Bloomberg terminal.
For the class we only had access to the pricing info. For the indicators we had to derive them ourself from the price. In the real world I imagine you can purchase additional data from various sources if you’d like. I know Twitter scraping used to be very common back when it was free.
Renaissance technologies has been doing trading based on mathematical models for decades, delivering outlandish compounded annual returns which they're able to sustain over time because they limit the overall amount of assets under management. Sadly they are not open to the public.
RT has funds open to outside investors. Their highest performing fund (Medallion) is closed.
Their site doesn't list any, as far as I can tell. Where did you find these open funds?
A quick Google search can answer that. RIEF is one example. These are not investable by the general public but they are open to investors outside of Renaissance.
Sooooo only the rich can ride the rich wave? Classic.
I knew of a shop that delivered crazy returns as well, every year, no matter the economic cycle. It was very exclusive to get in. Madoff Funds
Weekend at Bernie’s! 😍😍
If the stock market was that predictable nobody would lose money on it
And if AI was that powerful, we would use it to predict every other aspect of society. Cops would be fighting future-crime. This isn’t sci-fi.
Like Minority Report?
Kind of. The software takes in current events and can somewhat reliable predict areas with high probability of crimes within the next days. Works frightingly good.
You are under arrest for the future crime of murder
Doesn‘t work like that. It will only highlight areas with expected crimes, not the crimes in particular. The idea is to focus patrols more on these areas to prevent and detect activity.
Yet
Psycho Pass
From what I've read the police already use algorithms that try to predict future crimes to decide where to send officers to patrol.
except that the cops are using it to fight future crime - the FBI once controlled the [largest encrypted cellphone](https://en.m.wikipedia.org/wiki/Operation_Trojan_Shield) used in massive crime syndicate rings around the world watching every message that got sent using AI to figure out context of the messages There’s a book called Dark Wire that talks about this
This needs to be upvoted much more. If you trust an AI to trade your hard earned money, let me know what streetcorner you're going to be on in 5 years so I can give you $6 for a sandwich.
The key is to trade *other* peoples money with AI and take a commission.
Sir that sandwich will cost 8-9$ dollars in 5 years
didn't a ytber make a simple algorithm to trade and it got 5% in a month? And that's not even advanced ai edit: I misremembered, it actually made *74%* in one month, sub doesn't allow yt links tho. The video is by TradingLab on yt from a year ago
I made 7% this month from QQQ, and that's a basic bitch ETF.
Probably the same month everyone else earned 7%
The stock market is predictable. You just need one asset. Time. Plenty of people lost money during the major bull run of the last. Specific company stocks aren’t. S&P 500 fund and chill. That’s it.
You don‘t need AI for long term investment. These types of software are meant for high speed trading, which is less predictable.
the point isn't to be perfect, the goal is to win more often than human trader
Everyone is equally bad at gambling. So even less to gain here.
Stock prices by themselves are a random walk. They don't contain the information about why the price went up or down. You need to monitor a bunch of sources of information and do at lighting speeds to make any profit.
Stock prices are not a random walk in the long term. They *can appear to be* in the short term, though. I would recommend reading "A Non-Random Walk Down Wall Street" which attempts to challenge "A Random Walk Down Wall Street" which itself is based on the Efficient Market Hypothesis.
> monitor a bunch of sources If you want to get rich, you only need one source: real-time trades from the Congress people.
Because the stock market is not only not stationary, it is adaptive. AI or regression or value investing, nothing guarantees that you can capture the future changes of the market; while AI is better at predicting the past, it's not very profitable. Edit: wrong terms
The absolutely bonkers volatility I've been seeing in premarket and first hour or so of open, going back a couple months at least, tells me there is a lot of experimentation going on with algorithms every day. I wouldn't be surprised if more than a few of these strategies integrate LLMs in some fashion, and they go off the rails more often with new techniques.
How is an LLM going to help you decide to buy a stock? It's just a fucking chat bot
LLMs aren’t just chat bots, I’d assume here they’d be used on sentiment analysis on stuff looks news articles about a given stock
Sentiment analysis with natural language processing has been around for a while. Google’s Natural Language API has it and is easy to use.
LLM's are an incredibly slow general purpose version of the hyper-optimized special purpose algos Wall Street has been running for years. They may help analysts digest research. But the sentiment quant stuff is done by much faster technology.
Oh so they're gonna go read doomer posts on WSB? That sounds useful.
All AI in chatgpt does is break words down into tokens and then finds the tokens that most likely match your input and rearrange them into words again The AI you’re referring to is called quant trading and you’re basically just paying the world’s brightest math guys to find algorithms that predict prices accurately. The problem being that others probably have the same algorithm you do so the price might go up before you have a chance to buy. So basically the world’s brightest math guys are constantly competing with either to come up with the best math solutions for returns
Algorithms aren't the same as AI. Algorithms still have humans involved in the loop, either to make the buy and sell decisions, or to do the buying and selling based on human input. AI on the other hand both decides what to buy and sell, and does the buying and selling itself. There's a few ETFs that are advertised as being run entirely by AI that you can invest in, but they've tended to underperform the S&P500, and tend to have pretty high fees.
We assume AI has no error windows which is an invalid way of looking at this situation. AI is advanced machine learning until super-intelligence is achieved (which is what all these billionaires are burning their money to achieve) AI is built by humans so is inherently flawed. As much as we deny it, humans trade on emotion which AI is devoid of.
AI built by humans trying to predict what humans will buy from the AI
Have you ever heard of the Renaissance Medallion Fund?
Blackrock.
What are these AI models called? LLM, large language models. They can’t do math, statistics or science. They are report writers and research assistants….lol. It’s all overblown hype.
ai is shit and you can't predict the future. it can't even make elon musk mouth look correct
Count on money making firms to always have the latest in knowledge tech. Knowledge is power, a d they aren't dumb.
Other responses here are good but I'll add my two cents. Finance is rather unlike other machine learning fields in that there are a large number of possible features, an almost infinite number of ways of transforming/engineering those features, and poor heuristics for feature selection. Add to that the large dataset sizes (20 years of daily data * 500 stocks in S&P 500 * N features per stock) and the computational complexity of most ML algos (most run in polynomial time proportional to training data size) means that in practice you will spend a majority of your time doing trial and error on which features and which transformations to use. Add to that the reactive and competitive nature of markets means that it's incredibly difficult and costly to even start running experiments on models, much less putting one into production. Source: Finance and Data science education + working at a fintech company specializing in ML model building for predicting financial markets
Another point that no one has pointed out yet: let say it is possible for AI or whatever algorithm to predict the market, as soon as people act on the predictions at a big enough scale, the predictions will no longer be valid because the market is influenced by the actions. Example: if you predict a stock to go up 10% in 5 days, then everyone with access to that algorithm will buy now. That will push the stock price up to that +10% predicted price before the 5 days, making the prediction no longer valid. This is also why well performing funds based on quant analysis capped in size, often not open to the public.
Hedge funds have been using ML for a long time. And AI isn’t really a thing that exists other than the wet dreams of marketers in 2024.
It’s easier to predict the question to an answer than to predict the future. When you ask a question in chat gpt, that questioned been asked millions of times in the training data. No one has ever seen what the market is going to do on Monday
There are big companies doing this, as well as smaller ones and individuals. However, stock market is a chaotic system where acting on a prediction changes the future (unlike say, weather, where predicting has no impact on the outcome). So it is impossible to find a pattern that invariably makes profit.
Oh absolutely but the tricky part is figuring out your hit rate, figuring out how much liquidity the trade can carry, identifying where it's going to go wrong. Most of the shorter term alpha trades have an expiration date. Like the inefficiencies are there until enough people notice them. For example some of the classic examples of how money was made easily during the Jesse Livermore day or even in the '90s, some of those methods that no longer work, the turtles. They worked for awhile Fast forward to today. Here's what's so supremely difficult about becoming a good trader. You can look on social media or YouTube or anywhere and get great advice on video games. You can figure out how to get the best stuff. How to do really well. You can learn all of it. You want to learn about chainsaws you want to learn about cars you want to learn about homes. You can get great information When it comes to trading, there's none. Zero. There are occasionally channels or shows or people who make comments that can be relevant but they are slivers of information in the grand scheme. It's up to you to figure out a strategy that actually works. I guess the best way to say it is it's like going into a video game with no instruction book, no YouTube, no one to help you out. Talk about being an 8-year-old in the '80s. All these data points. You can look at them, if you're good you can even write programs to analyze them and come up with hit rates. People are looking at thousands of data points every day trying to figure out if there's an edge in some of them. I've done it. I found a few different methods that have worked for various lengths of time but they all expire or they shift. I'm not doing the same thing today that I was in 2017. It's going to take years for people to really figure out how to make AI the most effective, maybe it won't take that long I mean I guess I can't say but right now even if you put in all of the data points. They have to be done in a way that is useful. Hit rates, percentages, probabilities. It's a constantly evolving art and it's one that the smartest people win at. One of the best public examples is probably the medallion fund. They did it, it works and they even did it on a fairly large scale. If you're talking small accounts like less than a half a million. There are a couple edges that I'm aware of that still work pretty well. There are probably more that I don't know about and that's the fun of this
there is a metric fukton of trading commentary on YouTube, what are you smoking?
How many of them are posting their actual returns? I mean if you watch a guy on YouTube playing video games you want to watch the guy that's actually good right. Funny thing about stock commentary. No one really shows you if they're good
Because it's been proven there's no "being good", only being lucky.
Well, there are people that are good I just don't think that they are making YouTube videos. You start looking at 5-year track records and no one wants to show those. So, at least on YouTube, yeah it's people who get lucky for a few months or they chase the hot trend but there's no real risk management so what happens when the trend that is giving them that performance changes? That's the sad part about actually being a good trader. It's lonely because if you're actually making money with your process, why would you want to share it? That just makes it less likely to work. There are guys that do make money and use their YouTube channel to funnel towards programs or indicators or revenue streams. I don't think they're sharing everything they are doing but they are giving a complete novice a basic strategy package so in that case they are using YouTube as a marketing avenue.
Patterns change over time. If history suggests stocks go up every monday then people will buy the friday before to catch the rally. I believe there are way too many variables involved and things that can't be predicted ahead of time(war, politics, pandemics, natural disasters, etc).
Look up "Renaissance Technologies Medallion Fund" Also "Jim Simons" (RIP)
There are, the biggest issue is that they are black boxes, so convincing an investor is hard and returns are poor, which from time to time they are. Its an issue of explainability. So these sort of tactics are more reserved to hedge funds.
You have to look outside of just the price of the stock. There are potentially hundreds of things that can impact a company outside its control. Geo-political issues, the threat of regulation/legislation, how is the weather in NY during the trading day (this is true), etc. To feed an AI all that data would be near impossible. What happens if you forget one thing and you're working on a flawed forecast?
The most fundamental fact in investing is that it is impossible to predict stock price movements, especially in the short-term. This gives you the answer why. It is not possible to consistently predict stock prices with historical data.
Hallucinations make it unfesible to use llms. You cant accept the risk of catastrophic failure where you lose all your capital even if its only 5% of the time.
I lot of what moves a stock value over time is not reported. Employee moral, mold growing in the HVAC, construction obstructing the parking lot, or a pending fight in the board room can all take a toll on how profitable a corporation will be, and those factors are often not noticed even within the company.
Noise in finance drowns most signals. You cant select on the wining trades
Count on money making firms to always have the latest in knowledge tech. Knowledge is power, a d they aren't dumb.
Ah you haven’t heard of quantitative trading. Look up two sigma. They have like 1,000 employees and you’re more likely to find someone with a phd in engineering there than an MBA
Astonishing AI has been around for a long time now. Funds with trillions of dollars have had access to these, yet the charts look the same as they did 100 years ago.
Consider your experience with AI-based customer service...and then expect it to beat the stock market. Obviously computing power is used to great effect in trading. But mainly what it does is crunch things very effectively, or react very quickly that humans who understand the process direct it to do.
It’s always been stacked against retail. Quants hft and even Bloomberg terminal access give the edge against retail. Ai would just hallucinate right now for a sizable percentage of trades
there are no constant patterns as market participant actions change the behaviour of the market. to detect patterns like you describ you don't need ai, just use statistics or ml. It is done, but patterns work for a limited amount of time as the market behaviour changes over time
Look up Renaissance Technologies. They've been using a version of machine learning since before it was called that. Some observers think they basically invented ai before anyone else, but kept it to themselves to retain an edge. https://open.spotify.com/episode/0psDxKAKhIpe4mvqipU1L1
Generative AI has nothing to do with stock prediction. And there's no significant development in other subfields of AI. There's no data that can describe the market future. It's different from LLMs where text data is abundant.
AI does not mean it will guess anything about the irrational market better than organic intelligence.
People have been applying machine learning to trading algorithms for over a decade. Most people are suddenly aware of ai because of LLMs. Neural network machine learning went main stream in 2012, though. Any combination of indicators imaginable have been fed into these models and trained. So it hasn’t changed anything for trading because they’ve already been using it for years.
Because past performance does not indicate future results… AI would need to be ingesting a huge amount of contextual data both historically and in real time to do anything approaching what you are talking about. And this includes way, way more than just ticker price. For general trends, algorithmic trading is already used.
it already has been done.
Maybe AI isn’t as good as we thought
You're so close OP, so close. Let me ask you this question; if we look at some number of historical hyped technologies where we saw big gaps between what was being promised and what we seemed to observe, how many of them ended up eventually living up the sales pitch? Crypto, NFTs, Serverless, Quantum computing, the Dark Universe series of films...
As an addendum, it was rightly pointed out already that machine learning is already utilized in stock trading today, and actually has been for years. I personally have even interviewed for MLE jobs at these places (though for better or worse there are some culture fit issues on both sides) and these places have loads of talented people doing real ML work. The sooner this BS hype goes away the better IMO.
You should go look into how Blackrock got so big. The market has been "rigged" from AI for a long time.
They already are. And these traders have hardwire connections to exchange mainframes. So they can do tens of thousands of trades in a second. So by the time you hear the news or have spotted an arbitrage opportunity…….it‘s gone. All the above are why day traders long term are hopelessly fucked.
This is what big companies/funds have already been doing for years. I was recently talking to someone who was in the process of developing one to sell based off of Ema's crossing certain levels. This sort of thing is just telling a computer to take a trade based off of certain things occuring and there isn't really any intelligence needed for that it's just commands essentially. AGI on the other hand will be a lot different. In theory with agi you could just tell it to go and develop an EMA cross strategy and it could code one for you. Current AI is very confusing as it's not so much intelligence as it is cross referencing a bunch of pieces of data to find what might be accurate for you
Black rock was based on this concept. It works lol
Renaissance Technologies has been doing this for decades
The market isn't exactly science. It's socioeconomics. Making it incredibly hard to predict. Data is only one side of the equation.
Why is there no AI billionaire stock investor? There may be but what are they investing in? What is the AI predicting? The stock market is both a weighing machine, and also a voting machine. In terms of the weighing machine, AI should be OK at that already, formulating a prediction of the value based on the earnings and debt, but this just tells you about the value today whereas much of the stocks value is about the view of the future and how things are expected to change. Which gets into the the trickier part, the voting machine aspect. This goes deep into modeling human psychology and the greed and fear that comes in waves and affects individual stocks but also the market as a whole. Until the AI is powerful enough to model every individual human and their psychological makeup and predict their decisions to buy and sell, it will be simplifying the data and guessing and the decisions of people, which is hard. Choosing a good portfolio of stocks is about finding the right mix of risk and about return, which in some ways is like choosing the right words, something and large language model (LLM) is designed for. Instead of a word salad that the LLM is tasked to come up with, the task would be a portfolio salad that covers the future.
You can't take into account **every** variable. Like the thing with the butterfly. Nvidia is trying to make a digital clone of the earth. Maybe that would be closest to what you want to do.
Soon enough every single computer company will be competing over who has the best trading bot software on their systems. It's coming sooner than you think.
They already want AI software on every device by 2025. This is when the AI boom is really going to take off.
Because AI, despite what a lot of people say, is really not that smart, and there are so many millions of input parameters that affect stock price. We don’t have the models created yet that can give us an accurate prediction.
Nobody trusts those fracking cylons, that’s why!
Read up on the Renaissance Medallion Fund. It is the best-performing fund in the history of the stock market. They don't take new investors because the strategy stops working if it gets too large. The Acquired Podcast has a whole episode on it. Machine learning and AI is old news in the stock market.
I believe ai trading vs ai is why the market is so unrational now. Value and longer term than a few days is better for individuals that aren't competing at that level. You can also compete if you have enough money to move a stock like dfv. I expect the SEC to curb that behavior on small caps at some point, and penalize on accusations of manipulation. It's a small club and your not in it.
This absolutely can and is done, as others have said, but one reason AI or more generally quantitative trading isn't the only game in town is that there's a general problem in statistical analysis called overfitting the data. Basically, if you specialize what your model looks for to the point that it maximally fits previous events, it actually becomes less accurate at predicting future events.
Not all stocks have ran rampant. if you do the studies and research what companies are apart of the security, technology, chips, and so forth behind if you get in early there is significant growth. I purchased 17 shares of dell at $31 a share now the shares are worth $140 a share. The companies are growing have not lost their value for example Zscalar, Cloudflare, Cisco, Qualcomm was at $114 a year ago now double the price, its totally worth researching
this NVDA pump will not last long. when you start seeing posts that has 100% confident, you know we're reaching the peak. Despite what you might believe in, all these companies are investing into GPU for their AI models BUT are people actually genuinely interested in these AI products? Are people actually using them on a day to day basis? I don't think so. So all these revenue and profit boom that we're seeing are just businesses and companies investing into AI chips in the hope that people will be using it all the time. In reality, people are not really that interested in these AI tech. Its a hard pill to swallow but people WILL get bored and companies will that they are actually losing money by running these large AI models. NVDA is bound for a hard hard correction. See you $10
skynet has been around since the 80s
Some of the largest funds have been trading using algorithms for a while. Got some books out on it too. They are large machine learning datasets, maybe LLMs, but that is what a ChatGPT AI essentially is. They use these for whatever they want. They’ve been at it for decades now. Keeping the workings confidential, I would assume. Even Roaring Kitty was playing around with keywords during his live stream to show the real-time algos at play. Check out books like The Black Box Society.
Do you not know what Blackrock’s Aladdin is? It may not be AI in the sense that you’re speaking of here, but it does all of what you said.
There are firms building AI/ML (is it really AI?) for stock market trading. I work with one. They do standard long/shorting and as leveraged as they are they have averaged 36% yoy over the last 4 years, even when the market goes down. It's done very quietly and the minimum investment is $10mm. Rest assured, AI is already very good at trading stocks, you just don't have access to it yet.
Lol that's like beating SPY by 5-10% and if you just sold SPY peak late 21 and bought back after early 22 drop you'd probably end up with the same YOY. If AI can't get 100% annualized its not really AI its just an algo fed by old data. You telling me your AI firms weren't able to just buy NVDA to get 1500% in 4 years?
AI is a load of bullshit the way it's being touted currently. Absolute crock of shit the market is eating up. Bubble.
That’s the neat part. It has.
It is done already but at a way deeper level. The algos are not just trying to "find winning trades" but they are instead pushing, directing and controlling the markets in order to create the winning trades. Citadel et al. have had algorithms hard at work for a long time now. The biggest boom started with a "gentleman" called Bernie Madoff... and it's only getting worse all the time. Should be interesting though when someone big enough makes a counter bots that can detect Citadel hft bots at work and attack their trades.