New answers tagged dataset
0
votes
How to get sentiment score for a word in a given dataset
One option would be to use a library like Textblob. It has a function to calculate the sentiment of a sentence and assign polarity score and subjectivity score. So you could also pass a word instead ...
- 1,241
0
votes
Accepted
How to set the same number of datapoints in the different ranges in correlation chart
You can bin your variables to prevent overplotting and make the output cleaner.
Here is an example from StackOverflow:
https://stackoverflow.com/questions/16947210/making-binned-scatter-plots-for-two-...
- 206
1
vote
How to use confidence labels?
Just a few ideas that can be done easily with these confidence scores:
Note that with only two possibilities of 1 and 0.66, these confidence scores are practically discrete. Thus you could design the ...
- 20.7k
2
votes
Accepted
When to split Test and Training data from the full Dataset
You should split the dataset in training and test set first, because in a real environment, where your model is deployed, you just don't have a test set, since test set is used to check the ability of ...
- 68
0
votes
What is the difference between Pachyderm and Git?
Git is designed for code.
Pachyderm is designed for machine learning assets: data, pipelines, and notebooks.
You can put machine learning assets into git. However, git will treat machine learning ...
- 16.2k
1
vote
The single CSV created by combining a large number of CSV files is too large to process. What options do I have?
The specific questions 1 to 3 are actually irrelevant: the problem is not about file format, it's about the size of the data in memory. So even with some smaller file format, the data would still need ...
- 20.7k
0
votes
Create new rows based on a value in a column
First of all, I think your dataset have some problem because if you use single quote like this '['1','2','3','4']' in the value for the key then python will show ...
- 1
1
vote
Accepted
Create new rows based on a value in a column
The easiest way of doing this is probably to first convert the dataframe back to a list of rows, then use base python syntax to repeat each row n times, and then convert that back to a dataframe:
<...
- 5,641
0
votes
How to combine and separate test and train data for data cleaning?
before concatenation of test and train data. add new column to train and test data called type. And after preprocessing separate them based on column type. Here is a sample code.
...
- 101
0
votes
How can I generate anomalies in a dataset?
Anomalies usually mean datapoints that are not making sense, so you can just insert random numbers between a given range.
Generating 10 random numbers:
...
- 11
0
votes
Sampling methods for Text datasets (NLP)
One option is to enumerate each piece of text and then randomly pick an integer from the list of the integers. This will proportionally sample the empirical distribution.
"How many samples?" ...
- 16.2k
0
votes
Return the gradient and y intercept (m, b) to create two lines to best fit the data
One option could be to take a hierarchical approach:
Separate the values into two clusters. Since connectiveness within a cluster is important, use an algorithm like spectral clustering.
Fit a ...
- 16.2k
0
votes
Generating artificial data to extend learning set
SMOTE can be used to resample any continuous dataset, imbalanced or not.
If you use the Python implementation of SMOTE, sampling_strategy can be set to 'all' which ...
- 16.2k
0
votes
What is the best way to read SQL dataset in to Tensorflow?
here is a link for the driver-installation on possible linux distros.
and another link with installation und connection examples.
my code looks like as following,
...
Top 50 recent answers are included