Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Website categorization

If i wanted to do website categorization is there any prepared toolkit like pretrained model, or open data for categories with keywords? For example google listed some categories here https://cloud....
0
votes
1answer
37 views

Minimum number of samples on a CNN

Related to this paper, could it also be applied to convolutional neural networks? I ask this because FNN and CNN share many characteristics This is the formula: $(N_e+1)\cdot N_C + (N_C+1)\cdot N_S &...
20
votes
6answers
95k views

Uploading images folder from my system into Google Colab

I want to train a deep learning model on a dataset containing around 3000 images. Since the dataset is huge, I want to use Google colab since it's GPU supported. How do I upload this full image folder ...
0
votes
0answers
14 views

How to compare between two datasets of lexical density?

I have two dataset from two different texts representing lexical density as a proportion based on a corpus. Both datasets are represented in the images below. Now, let's suppose I want to know which ...
0
votes
0answers
12 views

Creating a validation set on a benchmark dataset without a validation set

I am using a benchmark dataset, which is already split into 5 subsets = (a, b, c ,d , e) and, I am constrained by already defined train/test configurations. For example, if I am using the abc as train ...
1
vote
0answers
36 views

Generating synthetic data based off existing real data (in Python)

I am looking for an approach to generate synthetic data for anomaly detection. We have real data, but want to inject anomalies to battle-test the model (the real data is too limited for likely future ...
1
vote
1answer
17 views

SMOTE for multi-instance learning i.e num_rows(x_train) > num_rows(y_train)

I have an imbalanced dataset and I wish to predict classes(0 or 1). Sample x_train: ...
1
vote
2answers
1k views

How to find nearest neighbors in SMOTE

I am reading the original paper by Chawla and others for SMOTE. I am trying to understand how to generate these synthetic examples for over-sampling the minority class. The paper says: "Synthetic ...
1
vote
1answer
952 views

SMOTE and standardisation

I have an unbalanced dataset X. I split it between data and labels, then I standardize the data. Then I use train_test_split to split between train and test data and I output the result. Now I want ...
0
votes
0answers
32 views

Regression ANN getting high root mean squared error values after re scaling dataset

I have a data set on predicting solar power generation, I am getting root mean squared loos of 0.3196 on training set on scaled values, but when I inverse transform them my loss rises to 298 on ...
2
votes
1answer
35 views

Automation of finding a starting point of measurement in a large dataset

I am looking for a way to automatically find a starting point of rising in my signal in Python. The data are collected with the frequency 10k (0.0001 s each) so the differences between each point are ...
0
votes
1answer
20 views

About building database for persian speech to text project [closed]

In connection with the persian speech to text project on github, I am trying to collect the database from the audio and text available on the ganjoor site and Youtube Persian videos with Persian ...
1
vote
2answers
46 views

Transforming data points to match other data points

I have two sets of data points, set A and set B. Each point of both sets has two dimensions $(x, y)$. Consider the set $A$ to be the orange set on the left of the image below and $B$ to be the blue on ...
0
votes
1answer
36 views

Analysis of passed/not passed students by enrolled year

I am analysing some data from my University about successful students through years. The dataset looks like as follows ...
1
vote
1answer
160 views

How to build a database of image data for machine learning?

I want to build a database of image data for machine learning. But how should this be done? I'm assuming people don't just dump all of their image data into a folder? Do they use a relational database ...
2
votes
3answers
519 views

Combining Two CSV's in Jupyter Notebook

I want to combine both CSV files based on Column1, also when combined each element of Column1 of both csv should match and also each row or Please suggest how to reorder Column1 according to another ...
1
vote
1answer
18 views

How we can Identify Specific Feature from a larger amount of Dataset?

In Machine Learning, we need to play with any kind of datasets. In the Dataset, There are too many records and features, Some datasets had lots of features (sometimes it's called ...
1
vote
2answers
4k views

Gradient Descent or Normal Equation?

Suppose you have a dataset with m = 50 examples and n = 15 features for each example. You want to use multivariate linear regression to fit the parameters theta to our data. Should you prefer gradient ...
0
votes
0answers
24 views

Where can I get mapped ground-truth of Pascal VOC2012 in COCO classes?

I'm trying to make use VOC2012/2007 dataset into COCO2014 training set ? As far as I know that most of class (20) in VOC is also in COCO (80). I think I can write code to map from VOC class to COCO ...
0
votes
0answers
17 views

Selling windows for products

I have sales data of hundreds of products. How they are sold over the year in terms of units sold and revenue. Sales of each product varies largely from one another and their seasonality too. Data ...
2
votes
2answers
89 views

Predictive power of a dataset

I am reading a book on machine learning for undergraduate. I am actually confused on linear regression flexibility as the say: Occasionally, linear regression will fail to recover a good solution for ...
11
votes
2answers
45k views

How to store strings in CSV with new line characters?

My question is: what are ways I can store strings in a CSV that contain newline characters (i.e. \n), where each data point is in one line? Sample data This is a ...
0
votes
0answers
28 views

Proximity multi-dimentional arrays. Which algorithms are commonly used?

I'm new to data science so still learning so much. I've been searching for proximity algorithms but I'm not sure which are suited for multi-dimensional arrays. Any guidance would be greatly ...
0
votes
0answers
14 views

Network flow dataset labeling

I have a network flow dataset with 500 mio. records that looks similar to this (highly simplified) ...
0
votes
0answers
8 views

Size of data required to build instruction Embeddings

I am using different sizes of ARMv7 data to build instructions embedding using a conventional word2vec model which would output a numpy array of Embeddings that ...
1
vote
4answers
88 views

Train a model using multiple data sources

I have to train a classification model to predict if a customer will buy a product or not. I have multiple (eg. 3 or 4) data sources. The variable distributions among the different data sources is ...
1
vote
0answers
115 views

Sampling methods for Text datasets (NLP)

I am working on two text datasets, one is having 68k text samples and other is having 100k text samples. I have encoded the text datasets into bert embedding. ...
0
votes
0answers
22 views
2
votes
1answer
27 views

Interpreting confidence interval results for datasets

I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval. I selected a random sample and manually annotated the sample and ...
9
votes
1answer
8k views

How to split train/test in recommender systems

I am working with the MovieLens10M dataset, predicting user ratings. If I want to fairly evaluate my algorithm, how should I split my training v. test data? By default, I believe the data is split ...
0
votes
2answers
31 views

Is it bad to have a lot of one class of Data [K-NN classifier]?

I am trying to train a sklearn K-NN classifier on a labeled text dataset (in Irish). There are 5 classes, 0-4, but there is a lot of variation between how many there are in each class. What I have ...
1
vote
2answers
60 views

How to deal with mixed type features?

I'm working on a dataset with more than 2000 features. Most of the features contain both numerical values and categorical features. For example, in the feature represents how long has the user been ...
2
votes
2answers
2k views

Data set with multiple tables [closed]

I want to check an algorithm that learns the connections between different tables in a database. Where can I find datasets that are compound of multiple tables (i.e. more than 5) that have ...
0
votes
1answer
25 views

Tagged dataset with photos for race detection

Looking for the tagged dataset, because I would like to identify race by photo. I tried using the UTKFace dataset from Kaggle, but it outputs hispanic and Arab people on images as ...
1
vote
0answers
27 views

Generating artificial data to extend learning set

I have dataset containing 42 instances(X) and one final Y on which i want to perform LASSO regression.All are continuous and numerical. As the sample size small, I wish to extend it. I am kind of ...
25
votes
3answers
43k views

Data Science Project Ideas [closed]

I don't know if this is a right place to ask this question, but a community dedicated to Data Science should be the most appropriate place in my opinion. I have just started with Data Science and ...
0
votes
2answers
170 views

how to make moving moving graphs that vary in time like a video? [closed]

like this, like shown in this video.
0
votes
0answers
50 views

Machine learning applications in Architecture field

Can there be Machine learning applications in Architecture field? Example : New York City 100 years back i.e. Year 1920, every ten years after 1920 and today i.e. Year 2020 images of New York City ...
0
votes
1answer
110 views

Labelled dataset for NLP regression

I am looking for a dataset, where I could use NLP techniques to estimate target value using regression. For example, I could be give a few sentences that describe an accident, and the target value ...
5
votes
4answers
4k views

Missing Values in Data [duplicate]

I have experienced that most of the datasets contain missing values, which make our task bit challenging. Please let me know how to fill up those missing values in an efficient way? and is there any ...
9
votes
3answers
17k views

Public dataset for news articles with their associated categories

I am wondering if there are any public datasets of Google news with various news categories such as politics, entertainment, lifestyle, general news, sports etc. I want to use such dataset for topic ...
10
votes
3answers
1k views

NASDAQ Trade Data

I am trying to find stock data to practice with, is there a good resource for this? I found this but it only has the current year. I already have a way of parsing the protocol, but would like to have ...
1
vote
0answers
80 views

How to generate custom image dataset for object-detection?

I have to build a custom logo detector from e-commerce images. I am aware of object detection techniques like YOLO, SSD etc and could find many resources on how to annotate a custom object detection ...
2
votes
2answers
89 views

When a dataset is huge, what do you do to train with all the images on i t?

I'm using Python 3.7.7. I'm trying to load a lot of NIFTI images using SimplyITK and Numpy from the [BraTS 2019 dataset][1]. This is the code I use to load the images into a numpy array. ...
1
vote
1answer
7k views

Numpy array from pandas dataframe

I am new in using python for data science. What is the difference between selecting a a column with: df['name'].values and ...
2
votes
1answer
230 views

How do I read the cord_19_embeddings_2020-07-16.csv from the COVID-19 Open Research Dataset Challenge (CORD-19) on Kaggle?

What I'm trying to do I wanted to use the CORD19 word embeddings csv to map them to certain findings from the rest of the dataset, but as we can see there are no stings in the first column. The way I ...
0
votes
1answer
50 views

High probabilities of success for wrong predictions

I'm studying the behavior of machine failures in a production scenario. For this, I generated random data to form my imbalanced training set, consisting of categorical data, which indicate whether or ...
0
votes
0answers
17 views

Model selection Tensorflow for custom dataset comparing panoramic images vs regular images (Image segmentation)

I have a question regarding the use of a specific model such as Deeplab and how to create a custom dataset for it. Background To give a bit of background info, I want to compare panoramically stitched ...
2
votes
1answer
4k views

Weighted mean with summarise_at dplyr

I strictly need to use the summarise_at to compute a weighted mean, with weights based on the values of another column ...
0
votes
2answers
149 views

What's the meaning of precomputed features?

When i learn about deep learning, I found dataset with precomputed features form. Link (http://cs.stanford.edu/people/karpathy/deepimagesent/coco.zip). What's the different with usual dataset?

1
5 6
7
8 9
24