Questions tagged [beginner]

For questions which concern getting started in Data Science or any of its related subdomains.

Filter by
Sorted by
Tagged with
0
votes
1answer
21 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
23
votes
2answers
9k views

How to deal with time series which change in seasonality or other patterns?

Background I'm working on a time series data set of energy meter readings. The length of the series varies by meter - for some I have several years, others only a few months, etc. Many display ...
0
votes
1answer
22 views

What common/simple problem would work well as a web app? [closed]

Context I'm currently writing a simple tutorial to demonstrate a tool to data scientists and analysts that turns Jupyter Notebooks into web apps. Basically, it discusses setting up the web app as a ...
10
votes
4answers
897 views

What initial steps should I use to make sense of large data sets, and what tools should I use?

Caveat: I am a complete beginner when it comes to machine learning, but eager to learn. I have a large dataset and I'm trying to find pattern in it. There may / may not be correlation across the data,...
0
votes
2answers
39 views

Inspect false classified

Recently, I was able to train a simple classification algorithm (my first ML-Project) and I even got a pretty satisfying precision score. Now I am looking for a way to inspect, which datapoints in ...
2
votes
4answers
2k views

Is there any book for modern optimization in Python?

I was reading Modern Optimization with R (Use R!) and wondering if a book like this exists in Python too? To be precise something that covers stochastic gradient descent and other advanced ...
6
votes
3answers
3k views

What is a policy in machine learning?

While I was reading the paper "Grounded Action Transformation for Robot Learning in Simulation", I came across the term "policy". Could someone explain to me what that actually is (...
0
votes
1answer
31 views

kMean clustering for recommendation

I have a file with 50000 rows from a library platform. Each individual row saves a user, and shows the order in which the user, has selected. The books could be from various categories (e.g. roman, ...
5
votes
4answers
6k views

How to apply class weight to a multi-output model?

I have a model with 2 categorical outputs. The first output layer can predict 2 classes: [0, 1] and the second output layer can predict 3 classes: [0, 1, 2]. How can I apply different class weight ...
0
votes
2answers
735 views

Tips for checking data integrity / data sanity?

I've read a few vague articles and watched a couple of YouTube videos on data integrity and data sanity, but none of them have mentioned ways to actually check these on datasets. I am interested in ...
2
votes
4answers
45 views

I need to find Simplest exercises available for learning neural networks

I'm learning neural networks, and I'd like to give myself simple exercises and try to train neural networks to solve them. I know that classifying hand-written digits is a classic problem, but I'm ...
0
votes
0answers
17 views

Where to begin with high throughput regression analysis with Python?

I have been doing some multiple regression on a medium sized data set. 5 predictor variables, about 150 data points. OLS, L1 and L2 regularisation. I've defined interaction parameters and added them ...
1
vote
1answer
39 views

Learning Resources for Recommendation system

Beginner here: Could you please suggest some of the learning resources (books/youtube/articles) for beginners who want to build a recommendation system for their organization. Have no clue about it ...
0
votes
0answers
13 views

Using lmer in R Studio with a large data set

I have a very large data set (127,000 observations, each point has 22 variables). I am only interested in 7 of those variables. The two dependent variables (linear and rotational acceleration) are ...
0
votes
0answers
22 views

What type of Machine Learning to use?

I have a beam loaded with random loads, for each timestep I obtain accelerations and maximum moments on the beam. My prediction should receive a timeseries of accelerometer data and output the ...
1
vote
1answer
39 views

How do I get started with machine learning and image recognition?

I'd like to get started with machine learning, specifically image recognition. I know that Python is the most popular language for ML since it's easy to pick up and there's tons of libraries for it. I ...
1
vote
0answers
12 views

Should I use a numeric scale or like/dislike for a ratings and recommendation engine?

I've very new to data science and am not sure what terms to even search for, so please forgive any ignorance. Let's assume that I'm building a movie recommendation engine. A user will rate a subset ...
58
votes
3answers
72k views

RNN vs CNN at a high level

I've been thinking about the Recurrent Neural Networks (RNN) and their varieties and Convolutional Neural Networks (CNN) and their varieties. Would these two points be fair to say: Use CNNs to break ...
0
votes
1answer
43 views

Which Machine Learning methods would be good for a beginner-level Data Science project for described dataset? [closed]

I want to do my first beginner-level project regarding Data Science. I picked my data set which is described below. I want to make a Python script which will make some predictions on our data set ...
2
votes
0answers
49 views

How to deal with large datasets? [closed]

I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even ...
0
votes
2answers
50 views

Finding Criminal Name in news?

We have news URLS, which we want to classify into crimes or non-crimes and further identify criminals by using NERs. For creating a model that identifies criminals, we tried SPacy which gave all the ...
3
votes
3answers
8k views

Determine useful features for machine learning model

I am working with a dataset with hundreds of features. I wish to create a simple machine learning model using 7-10 features from the original dataset. My question is this: What quantitative metrics ...
2
votes
1answer
211 views

Why can't I use data augmentation with a pretrained convnet?

Reading Deep Learning with Python by François Chollet. In section 5.3.1, we've instantiated a pretrained convnet, VGG16, and are given two options to proceed: A) Running the convolutional base over ...
-1
votes
1answer
100 views

Inputting Data for Machine Learning in PyTorch

I have scoured the internet and documentation, but have not yet been able to find a simple explanation of how to load data and/or simply type data into pytorch. I am just learning this library, and I ...
2
votes
1answer
78 views

How to define a neural model as linear or non linear

TL;DR: in what sense is the model of a neuron seen in the image above nonlinear? In chapter 1, section 1.3 MODELS OF A NEURON of Simon Haykin's Neural Networks book, the standard model of a single ...
0
votes
1answer
40 views

On Python is the `i` variable defined inherently? [closed]

I am new to this whole landscape, please need some hand holding at first, I can't find a beginner section. Can someone please tell me if i is defined inherently in ...
0
votes
1answer
42 views

What is the difference between the value -99 and NaN in a data column?

I am new to data science. I was looking into some datasets and I saw some values like -99, which I discovered later that it means that there is a missing value. Does this mean the same thing as NaN? ...
1
vote
1answer
57 views

Getting into data science and NLP outside of university

I'm almost 40. Four months ago I resigned from work to rest from stressing work and change my carerr direction. It took me some time to understand what I want to do and only then I started to learn ...
-1
votes
1answer
156 views

'Feature' definition

Precisely, what is a feature? Is it an attribute/property name or its value? E.g. would features examples be "name", "adress", ...? Or "Dorothy", "123 YellowBrick Road"?? If it is a property name, ...
2
votes
0answers
85 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
6
votes
1answer
93 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
0
votes
1answer
32 views

My network optimises for mean_squared_error, but the predictions are useless

After going through the tutorials of tensorflow I decided to do my own project. For this I did the following: I have scraped movie reviews from a polish site called filmweb. I got a dataset o 5400 ...
5
votes
2answers
540 views

Data scientist light?

Im a 26 year old guy with an MBA and I work as an ERP system administrator. I have been interested in the field of data science for a while now. I've always liked statistics and various analytical ...
2
votes
2answers
12k views

What is the difference between the training and testdata set?

What is the difference between the test and training data sets? As per blogs and papers I studied, what I understood is that we will have 100% data set that is divided into 2 sets (test data set is 30%...
3
votes
6answers
1k views

Where to start, which books [closed]

What is a good book or resource to start learning Artificial Intelligence ?
0
votes
1answer
400 views

Prerequisites for openCV [closed]

I am looking to start learning openCV to perform a car detection program. Should I know data science and/or machine learning stuff before going for openCV? If not, where can I start learning openCV ...
2
votes
1answer
119 views

Representing state in Q-Learning

I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me: If you have a game where you ...
1
vote
2answers
197 views

Is Mac OS the most suitable OS for a beginner Data Scientist? [closed]

As far as I understand a data scientist should learn both Excel and Bash. And the golden mean is this case is Mac OS, because it has both. And later I can install Ubuntu on Macbook.
1
vote
1answer
384 views

xgboost performance with predicted values as input

I have predicted the probability of loss using different features. Now when I used this with a non-important feature to predict the probability of loss. The first one is very close. logloss was close ...
8
votes
5answers
1k views

What does it mean when people say a cost function is something you want to minimize?

I am having a lot of trouble understanding this. Does it mean you should not use the cost function very often?
0
votes
1answer
48 views

Train, test and submission files - what am I supposed to do with all of them?

this might be very beginner's question. I'm working on Kaggle's HomeCredit Default Risk problem which has among others dataset train, test and submission files as can be seen in the link provided. The ...
0
votes
2answers
551 views

How to handle continuous values and a binary target?

This is going to be a very beginner's question. I have a datset of continues features like LoanAmount, LoanDuration(multiclass?), ... ClientIncome, ClientFreeSources, etc. and a binary target whether ...
1
vote
1answer
67 views

Data Exploration Tips & Tools

I am pretty new to the Data Science world and I am looking for some advices. One of the tasks that I have to do, is to go alone at a customer place for few days/weeks and dig into their data lake to ...
1
vote
1answer
31 views

MOOC importance [closed]

My problem is that at our masters program we have already been taught R, Statistics, Neural nets, Python (pandas, sklearn, etc). Also my master thesis is directly related to Deep Learning and Computer ...
4
votes
4answers
800 views

Data Science Career: From Researcher To Data Scientist

I’m currently working as a Research Assistant in Computer Science, specializing in both Human-Computer Interaction (HCI) and Health Informatics (HI) fields. As part of my role, I collect data from ...
1
vote
0answers
33 views

What are the introductory mathematics courses that are most pertinent to machine learning? [closed]

To provide more context to this question, I have found the following list of courses in order to learn calculus: Course | 18.01.1x | edX Course | 18.01.2x | edX Course | 18.01.3x | edX I also have ...
2
votes
5answers
2k views

How to compare Timeseries Sequences?

I have multiple time series sequences and I want for each new time series to find the most alike old one. I found that I can use the sum of errors between points. Is this a good approach? Is there a ...
0
votes
1answer
54 views

Timeseries of odds in race - how to pick a model

Being new to AI/ML I'd like some pointers to where to begin. I got data from horse races. Specifically, I got the odds for each runner during the race - ca 5 times per second. ...
1
vote
0answers
139 views

very large difference between cross_val and (multiple) r2 model evaluation

I did submit my first kaggle kernel, on the avocado dataset kernel link, I treated it like I should predict the avocado price so I splitted the dataset in a train & test set, fitted the model and ...