I've seen several job descriptions for data science which include developing a novel algorithm to be a part of production environments. Can you give some input of what could be meant here exactly? Would they mean an algorithm that behaves somewhat of an ETL: getting the data, cleaning it, storing it, and running a known model on it? Or something more complex like building variations of known prediction algorithms? Some examples will really be nice, as I'm studying to go into this field.
I am no data scientist, only an aspiring one for two years, moving from my background in software engineering and mathematics. So I took some courses, had some interviews, read a lot on the subject online. My opinion:
New algorithms are developed in research centers and at universities. And even then, most algorithms used in companies are already developed, and just optimized even more. Don't get your hopes up or be afraid that you have to reinvent gradient descent backpropagation.
With developing algorithms they most likely mean data extraction, data cleaning, data preparation for reporting statistics and presenting graphs. Maybe programmatically, maybe just using tooling.
The presented data may give more insights in simple relations in the domain, and maybe insights in more complex questions that can be asked.
You may get to define data flows, get to compare and select machine learning algorithms and tune its parameters. And continuously evaluate model performance in practice.
In industry its usually variations (but important ones) of the ground ideas.
Look at this boosting timeline:
- (Ada)Boosting Formally by two profesors in 2003
- xgboost by DLMC Distributed Machine Learning Community in 2014
- lightgbm by Microsoft in 2017
- catboost by yandex in 2017
- +- all the variations in between that did not caught up
Building on "basic" idea they removed all the negative ones while modifying/defining new sequence of steps to be executed.
To answer your question. As long as you have some non-trivial (debatable what it is) new (or variation of) sequence of steps to be executed, with appropriate degree of generalisation you got youself an algorithm. So the type of new algo is dependable on the field you work in.
It is indeed extremely rare for someone to develop a novel algorithm to solve their problem. In my experience it is more important to understand the business domain, how to normalize the data and choose what loss function should be minimized.
But it is very valuable to have experience with various kinds of algorithms so that you can pick the right tool for the job.
If a job listing says that a person "must develop new algorithms" I'd read it more like "must develop new programs / software / scripts".
You might be interested in the annual Kaggle survey about the state of Machine Learning and Data Science.
Some key results related to your question:
The most commonly used algorithms were linear and logistic regression, followed closely by decision trees and random forests. Of more complex methods, gradient boosting machines and convolutional neural networks were the most popular approaches.
Some numbers from the report (2020):
Linear or Logistic Regression - 83.7%
Decision Trees or Random Forests - 78.1%
Gradient Boosting Machines (xgboost, lightgbm, etc.) - 61.4%
Convolutional Neural Networks - 43.2%
Bayesian Approaches - 31.4%
Recurrent Neural Networks - 30.2%
Neural Networks (MLPs, etc.) - 28.2%
Transformer Networks (BERT, gpt-3, etc.) - 14.8%
Generative Adversial Networks - 7.3%
Evolutionary Approaches - 6.5%
What you do in business is program algorithms to solve biz problems.
Most all common problems have algorithms and processes already. So what you would do is adopt and apply it to your particular situation.
Data science being the buzzword du jour, and very poorly understood by almost everyone as to what it is and how to use it, is more a fancy word to get you to apply your programming skills to databases in some fashion.
Often this will be some sort of data 'mining' that looks for patterns so that management can ASSume that correlation means causation and then use that as a rule to make decisions.
This is how the last wall street meltdown occurred. They made BAD ASSumptions that were close enough for most cases to ignore the errors in it but when that black swan hit they went bankrupt and the government had to bail them out with our tax money.