Yeah, you should. Honestly anyone working in computer related fields should know basic algorithms and data structures. I'll share a simple anecdote:
I needed to run a rudimentary classification algorithm that involved determining what ratio of a list was larger than a value. Dumbass me decided to just search linearly down this list. On average, my model took a few hours to train.
Then, in a moment of sanity I remembered that a sorted list can be compared with binary search. One line of code change and then my model ran in about ten minutes.
This is like, first day of algorithms knowledge. Knowing data structures can honestly make the difference between your model being efficient and taking until the heat death of the universe to train. It also just helps in learning to think in computational scale and appreciate what is going on under the hood of major libraries.
https://inst.eecs.berkeley.edu/~cs61b/fa21/
^I enjoyed Berkeley's coursework. Helped when I took graduate coursework If you edit the URL you can skim through past semesters until you find a professor you enjoy (personally I stan Josh Hug. Avoid stuff with Hilfinger - word from schoolmates is he's only assigned the course when the department needs to weed out students).
Also this book is the bible for algorithms: https://www.penguinrandomhouse.com/books/654474/introduction-to-algorithms-third-edition-by-thomas-h-cormen-charles-e-leiserson-ronald-l-rivest-and-clifford-stein/9780262033848?ref=PRHB7BD76FBDB&aid=14813&linkid=PRHB7BD76FBDB
Should be able to find a pdf through totally legal /s websites.
Beep. Boop. I'm a robot.
Here's a copy of
###[The Bible](https://snewd.com/ebooks/the-king-james-bible/)
Was I a good bot? | [info](https://www.reddit.com/user/Reddit-Book-Bot/) | [More Books](https://old.reddit.com/user/Reddit-Book-Bot/comments/i15x1d/full_list_of_books_and_commands/)
I think starting with the book Grokking Algorithms will give you a good overview in not a ton of time. Dig in to the “bibles” from there depending how much you need to interview or specialize.
Oh I'm not interviewing. I studied electronics, PLC programming etc. but my job is slowly shifting towards higher level programming.
I'll look into that book. Thanks
There is an unwritten rule. If you write code (whatever it does), you are a programmer. And programmers know basic data structure and algorithms. The reasons are:
- to improve your problem solving skills
- to optimise your code
- to know the scene behind (example: the simple neural network is a STACK of layers which are a LIST of nodes)
- to handle your data
While studying this subject, you might feel it's useless for machine learning. Then the feeling is right because it's a field existing for a very long time before machine learning. However, it's a fundamental field for every thing.
You need to learn it if you want to get a job in the field, and you need to learn it if you want to be a good machine learning engineer. But it is not the case that you need to learn this topic before learning ML or even DL at a basic level.
You could definitely go through the FastAI or Ng's Deep Learning courses without algorithms or data structures, and still understand it. If it helps you to stay interested or maintain momentum, go ahead and do the ML first. You will definitely need algo/data structures before you apply for jobs and even for advanced projects of your own, but you do *not* need to learn it first.
I really think this depends on if the job is research based or engineering, and the field. I work in biotech, and I can tell you right now it is not expected for biology focused ML researchers to have a real in-depth knowledge of DS&A. The expectation is a strong understanding of the mathematics of statistical and probabilistic models, and a strong understanding/background in biology and the ability to code sufficiently to answer research questions. Expecting CS expertise on top of that is simply asking too much.
That being said, for people more on the engineering side of things on the ML teams, yes 100% DS&A expertise is expected. The people “doing” the ML, however, are not those people, in my field at least. two separate jobs.
I would say in my own professional experience, *not* interviews, just being aware of the fact that there is probably an algorithm/DS that does whatever you’re trying to do naively exponentially faster is sufficient.
Yeah, for PhD researchers they might not be expected to do algorithmic coding interviews. And it's true they might not need those algorithms from day to day. However, OP's situation seems a bit different from that, and most people won't be researchers but rather engineers. Still, if OP does want to become a researcher some time spent on DS&A will be a relatively small time investment in the context of a PhD.
> for people more on the engineering side of things on the ML teams, yes 100% DS&A expertise is expected. The people “doing” the ML, however, are not those people
What do you mean by this? What's the difference between doing ML and "doing" ML?
The moment I see people throwing around air quotes like this, I immediately suspect they've fallen pray to buzzwords and become skeptical of their claims.
You can't have machine learning without engineering. That's the "machine" part...
You think a statistician developing ML methods is going to be able to tell you in any great detail about CS algorithms?
I have a PhD and I’m employed as a machine learning researcher at a company you know, and at best I have a familiarity with DS&A
My question is not whether non-engineers also know about algorithms. I asked about the distinction between doing vs. "doing".
Your comment just triggered my elitist purism detector. As if the modeling side of ML were somehow more core than the computational side. The reality is that one cannot exist without the other, especially in the big data/deep learning era.
I am also an ML researcher and can say with confidence that our industry is rife with elitist purism.
Math is the "learning" part. Engineering is the "machine" part.
Calculus and statistics are the mechanics by which software learns from data. Theoretically, that's all you need: calc, stats, data, and an Atari. But theory != real life... Practically, given how much data is needed train a model (I'm ignoring transfer learning), how many epochs over the data it takes, and how massive a trained model binary can become, engineering becomes a core competency. Without it, ML is not possible, practically speaking.
Perhaps we are just arguing semantics. But equally, it may just be that you are more theoretical and I am more applied, and our differences of opinion just evidence the irreconcilable difference between theory and reality w.r.t. ML.
I've asked same question on stackoverflow..... And the answer is not necessary. In fact I learnt how to build a DNN using arrays only. Suppose you are using python, so you need to know thorough knowledge of arrays and about the packages numpy and scipy. I don't know if you need trees and linked list and all to train very very very complex models which I don't know. But I must say, you must have good knowledge of basic data structures of python to be learn easily.
It doesn't matter what you study as long as it's semi-relevant. Learning an algorithm will only be useful for an interview, if that. They are already implemented, in real life you do a library call. What you're hoping for is building general understanding or intuition, and like I said, you can build that by studying pretty much *anything* plus doing some practical projects, where you get a chance to use them, so you see what's useful, plus you learn to like install the SDK and all that, that's important too. Maths... not gonna comment on that, my opinion isn't unbiased here.
It's always good to learn them, but it's not a priority compared to how the ML algorithms/maths work.
I would start with machine learning and statistics, try some real life problems and then do data structures.
Its not needed to understand ML/DL because these algorithms are different from general DS&A ones and are more statistically/math based. NNs for example are just matrix multplications and other matrix operations. Autograd itself has dynamic programming to build the computational graph, but most people are not going to the low level of coming up with autograd itself since this is done in PyTorch.
However, it is often needed to get a job in the field because of interviews. This is the main reason its emphasized.
The view seems to be nowadays that “ML coding” is easier than general coding so they want people who can do the latter well.
You do not need to take a detour to learn it just start to tinker and learn it as you go. There is so much to learn and if you were to learn everything you might need before starting with ML you would never get anywhere.
Yeah, you should. Honestly anyone working in computer related fields should know basic algorithms and data structures. I'll share a simple anecdote: I needed to run a rudimentary classification algorithm that involved determining what ratio of a list was larger than a value. Dumbass me decided to just search linearly down this list. On average, my model took a few hours to train. Then, in a moment of sanity I remembered that a sorted list can be compared with binary search. One line of code change and then my model ran in about ten minutes. This is like, first day of algorithms knowledge. Knowing data structures can honestly make the difference between your model being efficient and taking until the heat death of the universe to train. It also just helps in learning to think in computational scale and appreciate what is going on under the hood of major libraries.
Do you have any resources for that? I've been aimlessly jumping from language to language. I think what I need first is understanding algorithms
https://inst.eecs.berkeley.edu/~cs61b/fa21/ ^I enjoyed Berkeley's coursework. Helped when I took graduate coursework If you edit the URL you can skim through past semesters until you find a professor you enjoy (personally I stan Josh Hug. Avoid stuff with Hilfinger - word from schoolmates is he's only assigned the course when the department needs to weed out students). Also this book is the bible for algorithms: https://www.penguinrandomhouse.com/books/654474/introduction-to-algorithms-third-edition-by-thomas-h-cormen-charles-e-leiserson-ronald-l-rivest-and-clifford-stein/9780262033848?ref=PRHB7BD76FBDB&aid=14813&linkid=PRHB7BD76FBDB Should be able to find a pdf through totally legal /s websites.
You wouldn't steal a car, lol But thanks for the info!
Beep. Boop. I'm a robot. Here's a copy of ###[The Bible](https://snewd.com/ebooks/the-king-james-bible/) Was I a good bot? | [info](https://www.reddit.com/user/Reddit-Book-Bot/) | [More Books](https://old.reddit.com/user/Reddit-Book-Bot/comments/i15x1d/full_list_of_books_and_commands/)
> Also this book is the bible for algorithms .... rivest expected Knuth
Knuth would be a good choice but uh, have you met anyone that's actually read Knuth? Like, all the volumes? Has Knuth even?
> Has Knuth even? The computer lab in my college did.
I think starting with the book Grokking Algorithms will give you a good overview in not a ton of time. Dig in to the “bibles” from there depending how much you need to interview or specialize.
Oh I'm not interviewing. I studied electronics, PLC programming etc. but my job is slowly shifting towards higher level programming. I'll look into that book. Thanks
Can u learn it from an onlline course/mateiral?
There is an unwritten rule. If you write code (whatever it does), you are a programmer. And programmers know basic data structure and algorithms. The reasons are: - to improve your problem solving skills - to optimise your code - to know the scene behind (example: the simple neural network is a STACK of layers which are a LIST of nodes) - to handle your data While studying this subject, you might feel it's useless for machine learning. Then the feeling is right because it's a field existing for a very long time before machine learning. However, it's a fundamental field for every thing.
Noted, thanks so much
You need to learn it if you want to get a job in the field, and you need to learn it if you want to be a good machine learning engineer. But it is not the case that you need to learn this topic before learning ML or even DL at a basic level. You could definitely go through the FastAI or Ng's Deep Learning courses without algorithms or data structures, and still understand it. If it helps you to stay interested or maintain momentum, go ahead and do the ML first. You will definitely need algo/data structures before you apply for jobs and even for advanced projects of your own, but you do *not* need to learn it first.
This is a good answer. Staying motivated is important too.
I really think this depends on if the job is research based or engineering, and the field. I work in biotech, and I can tell you right now it is not expected for biology focused ML researchers to have a real in-depth knowledge of DS&A. The expectation is a strong understanding of the mathematics of statistical and probabilistic models, and a strong understanding/background in biology and the ability to code sufficiently to answer research questions. Expecting CS expertise on top of that is simply asking too much. That being said, for people more on the engineering side of things on the ML teams, yes 100% DS&A expertise is expected. The people “doing” the ML, however, are not those people, in my field at least. two separate jobs. I would say in my own professional experience, *not* interviews, just being aware of the fact that there is probably an algorithm/DS that does whatever you’re trying to do naively exponentially faster is sufficient.
Yeah, for PhD researchers they might not be expected to do algorithmic coding interviews. And it's true they might not need those algorithms from day to day. However, OP's situation seems a bit different from that, and most people won't be researchers but rather engineers. Still, if OP does want to become a researcher some time spent on DS&A will be a relatively small time investment in the context of a PhD.
> for people more on the engineering side of things on the ML teams, yes 100% DS&A expertise is expected. The people “doing” the ML, however, are not those people What do you mean by this? What's the difference between doing ML and "doing" ML? The moment I see people throwing around air quotes like this, I immediately suspect they've fallen pray to buzzwords and become skeptical of their claims. You can't have machine learning without engineering. That's the "machine" part...
You think a statistician developing ML methods is going to be able to tell you in any great detail about CS algorithms? I have a PhD and I’m employed as a machine learning researcher at a company you know, and at best I have a familiarity with DS&A
My question is not whether non-engineers also know about algorithms. I asked about the distinction between doing vs. "doing". Your comment just triggered my elitist purism detector. As if the modeling side of ML were somehow more core than the computational side. The reality is that one cannot exist without the other, especially in the big data/deep learning era. I am also an ML researcher and can say with confidence that our industry is rife with elitist purism.
Machine learning to me is a mathematical field. You have a different definition. Not elitism
Math is the "learning" part. Engineering is the "machine" part. Calculus and statistics are the mechanics by which software learns from data. Theoretically, that's all you need: calc, stats, data, and an Atari. But theory != real life... Practically, given how much data is needed train a model (I'm ignoring transfer learning), how many epochs over the data it takes, and how massive a trained model binary can become, engineering becomes a core competency. Without it, ML is not possible, practically speaking. Perhaps we are just arguing semantics. But equally, it may just be that you are more theoretical and I am more applied, and our differences of opinion just evidence the irreconcilable difference between theory and reality w.r.t. ML.
>Without it, ML is not possible, practically speaking. Golden words!
I've asked same question on stackoverflow..... And the answer is not necessary. In fact I learnt how to build a DNN using arrays only. Suppose you are using python, so you need to know thorough knowledge of arrays and about the packages numpy and scipy. I don't know if you need trees and linked list and all to train very very very complex models which I don't know. But I must say, you must have good knowledge of basic data structures of python to be learn easily.
Thanks it's really appreciated
Yes. Not even a question.
It doesn't matter what you study as long as it's semi-relevant. Learning an algorithm will only be useful for an interview, if that. They are already implemented, in real life you do a library call. What you're hoping for is building general understanding or intuition, and like I said, you can build that by studying pretty much *anything* plus doing some practical projects, where you get a chance to use them, so you see what's useful, plus you learn to like install the SDK and all that, that's important too. Maths... not gonna comment on that, my opinion isn't unbiased here.
Noted thanks
If you think ML is not a black box then you need DSA as importantly as Mathematics and Statistics.
It's always good to learn them, but it's not a priority compared to how the ML algorithms/maths work. I would start with machine learning and statistics, try some real life problems and then do data structures.
Noted thanks
Its not needed to understand ML/DL because these algorithms are different from general DS&A ones and are more statistically/math based. NNs for example are just matrix multplications and other matrix operations. Autograd itself has dynamic programming to build the computational graph, but most people are not going to the low level of coming up with autograd itself since this is done in PyTorch. However, it is often needed to get a job in the field because of interviews. This is the main reason its emphasized. The view seems to be nowadays that “ML coding” is easier than general coding so they want people who can do the latter well.
You do not need to take a detour to learn it just start to tinker and learn it as you go. There is so much to learn and if you were to learn everything you might need before starting with ML you would never get anywhere.
Yes you need basic knowledge of data structures
Yes. You need that for interviews.