Questions tagged [beginner]
For questions which concern getting started in Data Science or any of its related subdomains.
150 questions
0
votes
0
answers
25
views
Help with Memory Usage Estimation of Java Threads
I am a data science beginner and I am looking for some directions regarding my problem. I hope it is ok if I ask my question here.
My data set consists of the overall memory usage of a JVM every 30s ...
- 101
0
votes
2
answers
885
views
Email Parsing using Machine Learning
I am new to machine learning. I have a project in which i need to extract some data from email by parsing email using machine learning and would really appreciate if you could guide me with that. I ...
- 1
0
votes
0
answers
76
views
Preparation for data scientist interviews/job for final year PhD student in ML theory
I would like to check/ask if there is anything particular that I need to prepare for data scientist interviews. I am quite unsure/lost about the requirements since I am coming from academia (see my ...
- 101
0
votes
0
answers
25
views
To Become a Data Analyst?
currently I am in year4 and also working as a product owner who would want to shift the occupation path to data analyst in banking, but the problem is I don't know where to start. May I have some ...
- 1
0
votes
1
answer
78
views
Beginner clustering project, what are the input features and how do I analyze the data?
I am a beginner to data science. I have this dataset on natural disaster events in Afghanistan from 2016 - 2017. Columns:
REGION (ex. North, North West, etc)
PROVINCE_NAME (kind of like US 50 states)
...
- 55
0
votes
1
answer
27
views
Seeking guidance regarding course selection
I have completed Prof. Andrew Ng's Machine Learning Specialization a week ago and now exploring different competitions (Titanic, House Price Prediction, and from the Playground Series). I want to ...
0
votes
1
answer
54
views
Why is a RNN inherently better for Time series than normal NN?
Similar to this question but I would like further clarification.
I understand that in abstract, RNNs can process inputs recursively and feed some state of memory through the recursion to have a sense ...
- 3
0
votes
0
answers
33
views
Clustering with features of categorical type
I have a dataset of 17 features and class label for each datapoint. The description of dataset is as follows:
2 features contain values 0, 1, 2, 3
15 features contain values 0, 1, 2
class label ...
- 1
0
votes
0
answers
30
views
Emulate simple logical rules with ML
I have the following question. I have created a sample dataset based on a some simple boolean rules. The data describes the shipping cost for a store.
The features are :
order value (range between 1-...
0
votes
0
answers
33
views
What is wrong with my neural network
I have been trying to code a neural network for the last couple of days. I have come pretty far, but i am super stuck right now and am looking for some help to make progress.
I have the feeling my ...
- 1
1
vote
1
answer
21
views
how to chose between these models?
i have a regression problem so i tried some regression models in order to pick the best one (based on RMSLE) here are the results:
here are all the models = [
('LR', LinearRegression()),
('Ridge', ...
- 11
1
vote
1
answer
85
views
Group people driving together based on GPS data
I need to group together people driving together based on GPS data. Data are collected by mobile phones. From each user we receive them in batches every 10 seconds. Each batch have list of GPS data (...
- 11
2
votes
1
answer
33
views
Newbie questions: real-time clustering of messages
I'm very much a newbie in NLP, so please accept my apologies if this is an obvious question, the wrong place to ask it or any other error I could be making.
I am considering using NLP for some subset ...
- 121
2
votes
1
answer
67
views
Model from an aggregate
I’m in a place where we’re unable to train models on data due to GDPR. What I want is to predict people getting a job (y) given (x,x,x,x…) their employment type working full time or part time, work ...
- 153
0
votes
1
answer
60
views
Predict data using Pre-Trained Classification Model
I have pre trained classification model (saved as pickle file) to predict employee attrition.
My question is when I use new dataset to predict using Pickle file do I need do all preprocessing steps (...
1
vote
0
answers
23
views
Training, Validation, and Testing Data in Supervised Learning
I've come up with some simple definitions for training, testing and validation data in supervised learning. Can anyone verify/improve upon my answers?
Training Data - Used by the model to learn ...
- 13
2
votes
1
answer
240
views
Various models giving 99% accuracy for KDDcup 99 dataset for Intrusion Detection, is there some sort of data leak I am missing?
Student who is quite new to all this here. I am currently working with the KDDcup 99 data for intrusion detection using various ML models (and ANN). My problem is that I am getting 99% often for ...
- 21
0
votes
1
answer
269
views
How do I fine-tune model performance after the initial run? (Scikit-Learn)
I've just started learning regression using scikit-learn and stumbled upon a problem. For a given dataset, let's say that I've imputed the missing data and one-hot encoded all categorical features. ...
- 13
1
vote
2
answers
147
views
For a student who is a beginner in quantitative research and statistics, which is the better statistical tool to start: R or IBM SPSS? Why? [closed]
Currently, I am writing my research design. However, I am still indecisive on what statistical tool should I use for the data analysis. I tried looking up on the internet and there are disparate ...
3
votes
1
answer
122
views
I can't figure out how to improve accuracy for tweet sentiment
I'm doing a beginning attempt at tweet sentiment analysis (positive, neutral, negative). So far I have cleaned the data and used a BoW to get some feeling of the data (>2.5k tweets). I also made ...
0
votes
0
answers
73
views
I need help to write an essay about: probabilistic method || Occam's razor || mathematics in the 21st
I am interested in one of the master's programs in Data Science. In the application process I need to submit an essay of 1,000 words about one of the following topics:
Drawbacks of the probabilistic ...
- 101
1
vote
0
answers
42
views
What is the learning path for the role of NLP engineer a beginner should follow? [closed]
I am data science enthusiast and have interest in NLP. How should I develop my understanding for the domain and prepare for the role of an NLP engineer? What are the must have skills to master NLP? ...
- 13
1
vote
0
answers
32
views
Distplot 'a' parameter, does not make logical sense to me, can anyone explain?
Most sns plot parameters have names that make somewhat logical sense,
For example scatterplot has the x attribute to choose what to plot on the x-axis, and y for y-axis. Similarly kdeplot uses the ...
- 11
0
votes
1
answer
54
views
Looking for binary class datasets with high class imbalance, that also have intra-class imbalance in the minority class
Newbie question alert...
For a college project I want to compare a few variants of SMOTE in terms of how much they improve classification of the minority class, over using random oversampling.
I have ...
3
votes
2
answers
817
views
Is NLP suitable for my legal contract parsing problem?
My company has a product that involves the extraction of a variety of fields from legal contract PDFs. The current approach is very time consuming and messy, and I am exploring if NLP is a suitable ...
- 31
0
votes
1
answer
12k
views
How to get R Table Column Names
I could not find anywhere how I can do this seemingly simple task (Retrieving column names in R was not helpful).
I have an R table defined as ...
- 121
0
votes
1
answer
27
views
What common/simple problem would work well as a web app? [closed]
Context
I'm currently writing a simple tutorial to demonstrate a tool to data scientists and analysts that turns Jupyter Notebooks into web apps. Basically, it discusses setting up the web app as a ...
0
votes
1
answer
39
views
Model with 2 datasets: combine time series data and statistics
I am new to data science modelling so apologies if using wrong terminology in advance.
I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
- 1
0
votes
1
answer
75
views
kMean clustering for recommendation
I have a file with 50000 rows from a library platform. Each individual row saves a user, and shows the order in which the user, has selected. The books could be from various categories (e.g. roman, ...
- 1
1
vote
4
answers
484
views
I need to find Simplest exercises available for learning neural networks
I'm learning neural networks, and I'd like to give myself simple exercises and try to train neural networks to solve them.
I know that classifying hand-written digits is a classic problem, but I'm ...
- 255
1
vote
1
answer
104
views
Learning Resources for Recommendation system
Beginner here:
Could you please suggest some of the learning resources (books/youtube/articles) for beginners who want to build a recommendation system for their organization.
Have no clue about it ...
- 199
1
vote
0
answers
32
views
What type of Machine Learning to use?
I have a beam loaded with random loads, for each timestep I obtain accelerations and maximum moments on the beam.
My prediction should receive a timeseries of accelerometer data and output the ...
- 11
1
vote
2
answers
304
views
How do I get started with machine learning and image recognition?
I'd like to get started with machine learning, specifically image recognition. I know that Python is the most popular language for ML since it's easy to pick up and there's tons of libraries for it. I ...
- 113
2
votes
0
answers
66
views
How to deal with large datasets? [closed]
I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even ...
- 21
0
votes
1
answer
56
views
Which Machine Learning methods would be good for a beginner-level Data Science project for described dataset? [closed]
I want to do my first beginner-level project regarding Data Science. I picked my data set which is described below. I want to make a Python script which will make some predictions on our data set ...
1
vote
2
answers
106
views
Finding Criminal Name in news?
We have news URLS, which we want to classify into crimes or non-crimes and further identify criminals by using NERs.
For creating a model that identifies criminals, we tried SPacy which gave all the ...
-1
votes
1
answer
481
views
Inputting Data for Machine Learning in PyTorch
I have scoured the internet and documentation, but have not yet been able to find a simple explanation of how to load data and/or simply type data into pytorch.
I am just learning this library, and I ...
0
votes
1
answer
42
views
On Python is the `i` variable defined inherently? [closed]
I am new to this whole landscape, please need some hand holding at first, I can't find a beginner section. Can someone please tell me if i is defined inherently in ...
- 109
2
votes
1
answer
281
views
How to define a neural model as linear or non linear
TL;DR: in what sense is the model of a neuron seen in the image above nonlinear?
In chapter 1, section 1.3 MODELS OF A NEURON of Simon Haykin's Neural Networks book, the standard model of a single ...
- 23
0
votes
1
answer
327
views
What is the difference between the value -99 and NaN in a data column?
I am new to data science. I was looking into some datasets and I saw some values like -99, which I discovered later that it means that there is a missing value. Does this mean the same thing as NaN? ...
- 231
1
vote
1
answer
94
views
Getting into data science and NLP outside of university
I'm almost 40.
Four months ago I resigned from work to rest from stressing work and change my carerr direction. It took me some time to understand what I want to do and only then I started to learn ...
- 11
0
votes
1
answer
828
views
'Feature' definition
Precisely, what is a feature? Is it an attribute/property name or its value?
E.g. would features examples be "name", "adress", ...?
Or "Dorothy", "123 YellowBrick Road"??
If it is a property name, ...
1
vote
2
answers
61
views
Inspect false classified
Recently, I was able to train a simple classification algorithm (my first ML-Project) and I even got a pretty satisfying precision score.
Now I am looking for a way to inspect, which datapoints in ...
- 11
2
votes
0
answers
307
views
How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]
I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material.
When discussing the training algorithm for a perceptron, the author states that looping ...
6
votes
1
answer
299
views
Mapping of categorical features into binary indicator features
I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
0
votes
1
answer
49
views
My network optimises for mean_squared_error, but the predictions are useless
After going through the tutorials of tensorflow I decided to do my own project. For this I did the following:
I have scraped movie reviews from a polish site called filmweb. I got a dataset o 5400 ...
- 101
0
votes
1
answer
871
views
Prerequisites for openCV [closed]
I am looking to start learning openCV to perform a car detection program. Should I know data science and/or machine learning stuff before going for openCV?
If not, where can I start learning openCV ...
0
votes
2
answers
3k
views
Tips for checking data integrity / data sanity?
I've read a few vague articles and watched a couple of YouTube videos on data integrity and data sanity, but none of them have mentioned ways to actually check these on datasets.
I am interested in ...
- 121
3
votes
1
answer
1k
views
Representing state in Q-Learning
I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me:
If you have a game where you ...
- 31
1
vote
2
answers
386
views
Is Mac OS the most suitable OS for a beginner Data Scientist? [closed]
As far as I understand a data scientist should learn both Excel and Bash. And the golden mean is this case is Mac OS, because it has both. And later I can install Ubuntu on Macbook.
user73561