10

Data science jobs cover a wide range of different activities so any answer is likely to be subjective. I'm in academia so my knowledge of the job market is limited, but from what I can see: The current context is very favorable to data scientists looking for a job, so anybody with some basic knowledge of ML has a chance. You're already above this level so ...


4

I want to know which model between additive and multiplicative best suits the above data. It is hard to tell just by looking at it. A multiplicative decomposition roughly corresponds to an additive decomposition of the logarithms. The additive decomposition is the most appropriate if the magnitude of the seasonal fluctuations, or the variation around the ...


4

I don't disagree with the other answers, but here's a different perspective you should bear in mind. Also, I can offer answers to your specific questions as someone who left academia (applied math/CS) for data science. In short, understanding the underlying needs and use cases for a business problem is paramount to any project, and so developing a strong ...


3

Calculate one day returns. Plot histogram of daily returns. Calculate $log(\frac{price_{i+1}}{price_i})$. Plot histogram of above logarithm. If second plot is more likely to be normally distributed then choose multiplicative model. Else, choose additive model. You can also perform statistical test for normal distribution and check, which one has higher p-...


3

Erwan nailed it (+1). But I think my addition is a little too long for a comment. You seem to be well ahead of where I was when I landed my DS job. I was in pure math, a couple of postdocs in, and had only a short time of self-study when I was applying for industry data science. On the other hand, I had actuarial exams in my undergrad, which probably ...


3

Just Don't take this advice blindly: The subjects you have mentioned in mathematics are core to solving problems using machine learning/Deep learning, programming is a tool to implement all this theory that you learn and on that basis you create your hypotheses and then test by implementing it in code for that you do not need coding skills of a coder you ...


2

Although not very helpful, the answer is probably "it depends". I like to do data cleaning and some EDA together since EDA can highlight appropriate treatments to clean the data - e.g. influencing how to handle missing values. I think data transformation should be done just prior to modelling; whether or not you need to do any transformation at all depends ...


2

I think you're quite confused. Hadoop is a collection of software that contains a a distributed file system called HDFS. Essentially HDFS is a way to store data cross a cluster. You can access file stores as you would in a local file store (with some modification) and modify things via Java API. Furthermore, ON TOP OF the file system there exist a ...


2

There is a very cool active Python package called pandas-profiling, is exactly what you want. With a simple pandas_profiling.ProfileReport(df) it returns a lot of important statistical information about your data, the official documentation says: For each column the following statistics - if relevant for the column type - are presented in an interactive ...


2

It seems like a challenging problem. If it were my task, I would start with a probabilistic approach like apriori, but you may want to check out Naive Bayes based approach. There are some differences in these approaches, but, either one may produce decent results. More generally, I think the analysis you want to perform is $Association\ Rule\ Learning$. ...


2

really hard answer for u question cause there is to little information. Try to make EDA and attach it to question. Cause EDA define the model Anywhere, for low corellated data, try to use k-NN. If u use Python: scikit-learn have implementation. Also u can try decision trees. Sorry, but i'm also new in DS and can be wrong:)


2

TL;DR If you have unlimited time and use a 64-bit version of Excel, you can get as far with Excel as any other data analysis tool. Time I mention time as my first factor, because Excel only has basic funcitonality built in, such as summing, random number generation, lookups etc. These correspond to a kind of standard library, which Python and R also have. ...


2

Definition A inner product (AKA dot product and scalar product) can be define on two vectors $\mathbf{x}$ and $\mathbf{y}$ $\in \mathcal{R^n} $ as $$ \mathbf{x.x^T} = <\mathbf{x},\mathbf{y}>_\mathcal{R^n}=<\mathbf{y},\mathbf{x}>_\mathcal{R^n} = \sum_{i=1}^{n} x_i \times y_i $$ The inner product can be seem as the length of the projection of a ...


2

Common use cases include: Fraud detection Transactions volume prediction Next transaction date Fraud detection This is usually tackled with anomaly detection. It requires information on the two transaction parties and using machine learning to figure out when a transaction is out of the norm and flagging as a potential case of fraud. Transactions volume ...


2

Either do unsupervised learning with something like k-means clustering or DBScan where you attempt to segment students into groups and see if you can discern any insights based on the cluster generated or pick a threshold for certain categories, create a class column and label each student, and do a classification model.


2

There is high variance within each group. Even though there is a mean difference between the groups, there is a high amount of spread within just treatment A or just treatment B. From a statistical point of view, the difference between the groups could be due to chance because of the large spread relative to the small mean difference. Due to the amount of ...


1

df['date'] = pd.to_datetime(df['created_at']).dt.date to_datetime: Convert argument to datetime. For example, if your column of "created_at" is a string column, it converts it to a datetime column dt: Access object for datetime like properties of the Series values.


1

I am also new to time series forecasting. I used simple lineplot to visualize the time series data. Some of the models used for time series are : ARIMA SARIMA prophet LSTM (deep neural networks) LBATS You can refer to this link to know what things you should consider while building a time series model


1

You can calculate mean target for each categorical variable and compare its values. In pandas this can be done easily: df.groupby('categorical_feature').target.mean() Then you can make a histogram to compare the approach. I also, seaborn has a catplot, where it do the same as above in a bar plot format, showing mean value for target variable based on each ...


1

Clustering can be an ideal choice here. From the question, seems the data will most probably be in continuous format. Essentially, clustering is a method of finding groups of similar objects. The similarity between the objects is determined by the type of distance measure being used. With this background, you can start by finding the videos that are most ...


1

How large is N? Can you reshape your data into something like: target i1 i2 i3 i4 i5 ... i9 ... iN 1 1 0 1 0 0 ... 0 ... 0 1 0 0 0 1 1 ... 1 ... 0 0 1 0 0 0 0 ... 0 ... 0 Once you have fit everything into a data frame, you can use any two-class supervised classification algorithm to build ...


1

Speaking as someone from a finance background, the `usual' model for a stock price process is $\frac{dS}{S}=r dt + \sigma dW_t$ i.e. we assume the returns (not the absolute price changes, i.e. dS/S is approximately the daily percentage change in price) as having a 'drift' equal to the risk free rate (the interest rate r) and a random shock $dW_t$ with ...


1

No, this is the classical application of statistical test. You should consider the problem in the scope of two-sample test. Machine learning does not fit in the picture.


1

I don't think someone can learn about everything in the image and be good at all of them. Especially when you get out of school ! You need to master: Fundamentals Statistics Programming Machine Learning Visualization The rest depends on what you want to specialize into.


1

From further research I've discovered that the frequency is given by the index of the FFT multiplied by the sampling rate and divided by the size of the array. And the amplitude is the magnitude of the complex number. So the full code for such a plot would be as follows # X is some set of Wait times between spikes, below is just an example X <- c(56, 3, ...


1

I can't add a comment so I'll post this here. My main question is why are your feature values so wildly different different between you your test and training sets? For example the raw values of feature 1. Training set: All values > 20, looks like they average about 35-40. Test set: All values around 2, looks like they all fall between 2-3. What is ...


1

The aim is to predict pm2.5 ( target variable ). Step 1: Data Cleaning. Remove unwanted features and fill the missing values. Step 2: To learn about the features, perform data visualization. You can plot a linear plot with TEMP and pm2.5 and see how it varies with change in temp. Step 3: The next step is to find the relationship between features. Some ...


1

Most of the classification algorithms can return a probability or a similar measure. Usually, you don't get black or white, but nuances of grey, and then based on a threshold, you select one or the other answer.


1

I had some interesting results conducting association rules analysis on website behavior i.e. which pages or content are looked at together and what can I learn from this in regards to improving content, user flow and even user classification. This is similar to a shopping basket analysis but instead of.purchased products you look at read pages.


1

Recommendation Engine is one such application of ML that can be used to extract the useful information from the user available data and offer them the product that is the most relevant to him. You can map this to classic classification problem where expected output of ML algorithm is to come up with the product that customer will buy.


Only top voted, non community-wiki answers of a minimum length are eligible