Questions tagged [beginner]

For questions which concern getting started in Data Science or any of its related subdomains.

Filter by
Sorted by
Tagged with
0 votes
1 answer
14 views

What kind of model would you use to predict recurring transactions?

Trying to learn about some DS concepts through the lens of this use-case. What model would you use to predict/detect events or transactions on a regular interval? Would this be supervised or ...
user avatar
0 votes
1 answer
16 views

Predict data using Pre-Trained Classification Model

I have pre trained classification model (saved as pickle file) to predict employee attrition. My question is when I use new dataset to predict using Pickle file do I need do all preprocessing steps (...
user avatar
0 votes
0 answers
12 views

Multicolinear Predictors Effect on Model

I know that multicolinear predictors in a model aren't ideal because it causes the model to be sensitive to very minor changes, which then reduces our ability to interpret the effects of each ...
user avatar
0 votes
0 answers
19 views

Beginner needs guidance. Machine Learning, preparing training data

i try to dip my feet into the field of computer vision and want to avoid mistakes along the way. The problem I have to solve: Classifiy images of 3D dental scans. For example: I wrote a script to ...
user avatar
1 vote
0 answers
16 views

Training, Validation, and Testing Data in Supervised Learning

I've come up with some simple definitions for training, testing and validation data in supervised learning. Can anyone verify/improve upon my answers? Training Data - Used by the model to learn ...
user avatar
2 votes
1 answer
38 views

Various models giving 99% accuracy for KDDcup 99 dataset for Intrusion Detection, is there some sort of data leak I am missing?

Student who is quite new to all this here. I am currently working with the KDDcup 99 data for intrusion detection using various ML models (and ANN). My problem is that I am getting 99% often for ...
user avatar
0 votes
1 answer
30 views

How do I fine-tune model performance after the initial run? (Scikit-Learn)

I've just started learning regression using scikit-learn and stumbled upon a problem. For a given dataset, let's say that I've imputed the missing data and one-hot encoded all categorical features. ...
user avatar
1 vote
2 answers
46 views

For a student who is a beginner in quantitative research and statistics, which is the better statistical tool to start: R or IBM SPSS? Why? [closed]

Currently, I am writing my research design. However, I am still indecisive on what statistical tool should I use for the data analysis. I tried looking up on the internet and there are disparate ...
user avatar
0 votes
0 answers
16 views

Group related items by their description and tag each group. [Pen, Eraser] : Stationary

So have a list of data similar to the table below. It will be captured by a chatbot so I expect natural language but in the form of a structured command: Add {Qty} {item description} to {location} ID ...
user avatar
  • 101
0 votes
0 answers
82 views

Does Bias always decrease when Complexity increase?

(I'm just starting learning about ML stuff and so please don't be rude if the following question is to stupid or totally wrong) I'm reading about Bias-Variance Trade off and I don't understand the (...
user avatar
3 votes
1 answer
29 views

I can't figure out how to improve accuracy for tweet sentiment

I'm doing a beginning attempt at tweet sentiment analysis (positive, neutral, negative). So far I have cleaned the data and used a BoW to get some feeling of the data (>2.5k tweets). I also made ...
user avatar
0 votes
0 answers
13 views

Tableau: Trying to determine the category of one table based on the dynamic aggregate of another table for a Tableau Dashboard

I have one table that contains unique rows for all my quote requests: Quote_ID 1234 1235 1236 1237 1238 in a second table that I've joined (1-0*) with a ...
user avatar
0 votes
0 answers
41 views

I need help to write an essay about: probabilistic method || Occam's razor || mathematics in the 21st

I am interested in one of the master's programs in Data Science. In the application process I need to submit an essay of 1,000 words about one of the following topics: Drawbacks of the probabilistic ...
user avatar
1 vote
0 answers
37 views

What is the learning path for the role of NLP engineer a beginner should follow? [closed]

I am data science enthusiast and have interest in NLP. How should I develop my understanding for the domain and prepare for the role of an NLP engineer? What are the must have skills to master NLP? ...
user avatar
1 vote
0 answers
28 views

Distplot 'a' parameter, does not make logical sense to me, can anyone explain?

Most sns plot parameters have names that make somewhat logical sense, For example scatterplot has the x attribute to choose what to plot on the x-axis, and y for y-axis. Similarly kdeplot uses the ...
user avatar
0 votes
1 answer
29 views

Looking for binary class datasets with high class imbalance, that also have intra-class imbalance in the minority class

Newbie question alert... For a college project I want to compare a few variants of SMOTE in terms of how much they improve classification of the minority class, over using random oversampling. I have ...
user avatar
3 votes
2 answers
388 views

Is NLP suitable for my legal contract parsing problem?

My company has a product that involves the extraction of a variety of fields from legal contract PDFs. The current approach is very time consuming and messy, and I am exploring if NLP is a suitable ...
user avatar
0 votes
1 answer
2k views

How to get R Table Column Names

I could not find anywhere how I can do this seemingly simple task (Retrieving column names in R was not helpful). I have an R table defined as ...
user avatar
  • 111
0 votes
1 answer
25 views

What common/simple problem would work well as a web app? [closed]

Context I'm currently writing a simple tutorial to demonstrate a tool to data scientists and analysts that turns Jupyter Notebooks into web apps. Basically, it discusses setting up the web app as a ...
user avatar
0 votes
1 answer
30 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
user avatar
  • 1
0 votes
1 answer
59 views

kMean clustering for recommendation

I have a file with 50000 rows from a library platform. Each individual row saves a user, and shows the order in which the user, has selected. The books could be from various categories (e.g. roman, ...
user avatar
1 vote
4 answers
80 views

I need to find Simplest exercises available for learning neural networks

I'm learning neural networks, and I'd like to give myself simple exercises and try to train neural networks to solve them. I know that classifying hand-written digits is a classic problem, but I'm ...
user avatar
1 vote
1 answer
60 views

Learning Resources for Recommendation system

Beginner here: Could you please suggest some of the learning resources (books/youtube/articles) for beginners who want to build a recommendation system for their organization. Have no clue about it ...
user avatar
  • 179
1 vote
0 answers
29 views

What type of Machine Learning to use?

I have a beam loaded with random loads, for each timestep I obtain accelerations and maximum moments on the beam. My prediction should receive a timeseries of accelerometer data and output the ...
user avatar
  • 11
1 vote
2 answers
139 views

How do I get started with machine learning and image recognition?

I'd like to get started with machine learning, specifically image recognition. I know that Python is the most popular language for ML since it's easy to pick up and there's tons of libraries for it. I ...
user avatar
2 votes
0 answers
55 views

How to deal with large datasets? [closed]

I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even ...
user avatar
  • 21
0 votes
1 answer
48 views

Which Machine Learning methods would be good for a beginner-level Data Science project for described dataset? [closed]

I want to do my first beginner-level project regarding Data Science. I picked my data set which is described below. I want to make a Python script which will make some predictions on our data set ...
user avatar
1 vote
2 answers
76 views

Finding Criminal Name in news?

We have news URLS, which we want to classify into crimes or non-crimes and further identify criminals by using NERs. For creating a model that identifies criminals, we tried SPacy which gave all the ...
user avatar
-1 votes
1 answer
385 views

Inputting Data for Machine Learning in PyTorch

I have scoured the internet and documentation, but have not yet been able to find a simple explanation of how to load data and/or simply type data into pytorch. I am just learning this library, and I ...
user avatar
0 votes
1 answer
40 views

On Python is the `i` variable defined inherently? [closed]

I am new to this whole landscape, please need some hand holding at first, I can't find a beginner section. Can someone please tell me if i is defined inherently in ...
user avatar
  • 109
2 votes
1 answer
171 views

How to define a neural model as linear or non linear

TL;DR: in what sense is the model of a neuron seen in the image above nonlinear? In chapter 1, section 1.3 MODELS OF A NEURON of Simon Haykin's Neural Networks book, the standard model of a single ...
user avatar
0 votes
1 answer
107 views

What is the difference between the value -99 and NaN in a data column?

I am new to data science. I was looking into some datasets and I saw some values like -99, which I discovered later that it means that there is a missing value. Does this mean the same thing as NaN? ...
user avatar
1 vote
1 answer
73 views

Getting into data science and NLP outside of university

I'm almost 40. Four months ago I resigned from work to rest from stressing work and change my carerr direction. It took me some time to understand what I want to do and only then I started to learn ...
user avatar
  • 11
0 votes
1 answer
535 views

'Feature' definition

Precisely, what is a feature? Is it an attribute/property name or its value? E.g. would features examples be "name", "adress", ...? Or "Dorothy", "123 YellowBrick Road"?? If it is a property name, ...
user avatar
1 vote
2 answers
49 views

Inspect false classified

Recently, I was able to train a simple classification algorithm (my first ML-Project) and I even got a pretty satisfying precision score. Now I am looking for a way to inspect, which datapoints in ...
user avatar
  • 11
2 votes
0 answers
303 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
user avatar
6 votes
1 answer
174 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
user avatar
0 votes
1 answer
37 views

My network optimises for mean_squared_error, but the predictions are useless

After going through the tutorials of tensorflow I decided to do my own project. For this I did the following: I have scraped movie reviews from a polish site called filmweb. I got a dataset o 5400 ...
user avatar
  • 101
0 votes
1 answer
768 views

Prerequisites for openCV [closed]

I am looking to start learning openCV to perform a car detection program. Should I know data science and/or machine learning stuff before going for openCV? If not, where can I start learning openCV ...
user avatar
0 votes
2 answers
2k views

Tips for checking data integrity / data sanity?

I've read a few vague articles and watched a couple of YouTube videos on data integrity and data sanity, but none of them have mentioned ways to actually check these on datasets. I am interested in ...
user avatar
  • 121
3 votes
1 answer
613 views

Representing state in Q-Learning

I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me: If you have a game where you ...
user avatar
1 vote
2 answers
305 views

Is Mac OS the most suitable OS for a beginner Data Scientist? [closed]

As far as I understand a data scientist should learn both Excel and Bash. And the golden mean is this case is Mac OS, because it has both. And later I can install Ubuntu on Macbook.
user avatar
3 votes
1 answer
391 views

Why can't I use data augmentation with a pretrained convnet?

Reading Deep Learning with Python by François Chollet. In section 5.3.1, we've instantiated a pretrained convnet, VGG16, and are given two options to proceed: A) Running the convolutional base over ...
user avatar
  • 69
1 vote
1 answer
104 views

Train, test and submission files - what am I supposed to do with all of them?

this might be very beginner's question. I'm working on Kaggle's HomeCredit Default Risk problem which has among others dataset train, test and submission files as can be seen in the link provided. The ...
user avatar
0 votes
2 answers
1k views

How to handle continuous values and a binary target?

This is going to be a very beginner's question. I have a datset of continues features like LoanAmount, LoanDuration(multiclass?), ... ClientIncome, ClientFreeSources, etc. and a binary target whether ...
user avatar
1 vote
1 answer
33 views

MOOC importance [closed]

My problem is that at our masters program we have already been taught R, Statistics, Neural nets, Python (pandas, sklearn, etc). Also my master thesis is directly related to Deep Learning and Computer ...
user avatar
  • 11
4 votes
4 answers
865 views

Data Science Career: From Researcher To Data Scientist

I’m currently working as a Research Assistant in Computer Science, specializing in both Human-Computer Interaction (HCI) and Health Informatics (HI) fields. As part of my role, I collect data from ...
user avatar
1 vote
0 answers
41 views

What are the introductory mathematics courses that are most pertinent to machine learning? [closed]

To provide more context to this question, I have found the following list of courses in order to learn calculus: Course | 18.01.1x | edX Course | 18.01.2x | edX Course | 18.01.3x | edX I also have ...
user avatar
10 votes
6 answers
16k views

How to apply class weight to a multi-output model?

I have a model with 2 categorical outputs. The first output layer can predict 2 classes: [0, 1] and the second output layer can predict 3 classes: [0, 1, 2]. How can I apply different class weight ...
user avatar
0 votes
1 answer
162 views

Timeseries of odds in race - how to pick a model

Being new to AI/ML I'd like some pointers to where to begin. I got data from horse races. Specifically, I got the odds for each runner during the race - ca 5 times per second. ...
user avatar
  • 3