Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
1
vote
1answer
20 views

How would you encode missing pixels in image data?

I am working through an example on the MNIST dataset, and was just curious, if your image input data were missing some pixels, how would you encode it. Since the values are always positive, and ...
0
votes
1answer
9 views

Which colour channel from a TIFF image do I have to use?

I'm going to use the following dataset to do semantic segmentation with U-Net network. LGG Segmentation Dataset This dataset contains brain MR images together with manual FLAIR abnormality ...
0
votes
0answers
14 views

Where to save image features to avoid repeating extraction?

I am implementing Bag-of-Visual-Words. Part of that is SIFT, it has a task of extracting features and outputs a vector. Afterwards, the vector are clustered and made into a histogram. Histograms are ...
1
vote
1answer
69 views

How to prepare Audio-text data for speech recognition

I have gathered some raw audio from all the conferences, meetings, lectures & casual conversation that I was part of. The machine transcription did not offer good results (from Azure, AWS etc.) I ...
1
vote
1answer
38 views

Creating a valid dataset for obtaining results

I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
1answer
20 views

Semantic segmentation with greyscale images

I'm trying to reproduce a research with greyscale images instead of colour images. I have found that there are pre-trained networks, like VGG16, with ImageNet. But that dataset has colour images, and ...
0
votes
0answers
42 views

Do I need to use learning algorithms for this type of data?

I'm studying the behavior of machine failures in a production scenario. For this, I generated random data to form my imbalanced training set, consisting of categorical data, which indicate whether or ...
1
vote
2answers
38 views

What are suitable datasets for univariate time series forecasting with RNNs, LGBM, TBATS, SARIMA models (topic, frequency, sources)? [closed]

I am currently looking for a suitable dataset (univariate time series) for short-term forecasting using lag features or moving windows of lag features to employ models like LSTM, GRU, SARIMA, LGBM, ...
1
vote
0answers
32 views

What is the correct way to use Active Learning on a huge and unbalanced text dataset?

I am working on a huge and unlabeled text dataset. It has over 30M lines (= sentences). We are trying to detect illegal sentences by analyzing 20 different legal issues (e.g. racism, insults, etc.). ...
1
vote
0answers
31 views

Fine tuning a automatic speech recognition model with my own dataset

I'm using wav2letter to develop a speech-to-text system. wav2letter has pre-built acoustic and language models which is great, however the audio that I am transcribing from is unique in comparison to ...
0
votes
1answer
28 views

Comparing Dataset - Should I use the same Test dataset?

I am training ML CNN model. I want to compare different images dataset. The dataset all have different characteristics (Translated or not, Rotated or not, etc.). I do not modify the ML model between ...
1
vote
1answer
15 views

What is the right way to store datasets for a CNN project [closed]

Our image classification project has thousands of raw photos, masks and reshaped images. We store source code in git. But datasets don't belong to source code version control. How should we store thee ...
1
vote
2answers
49 views

How to deal with mixed type features?

I'm working on a dataset with more than 2000 features. Most of the features contain both numerical values and categorical features. For example, in the feature represents how long has the user been ...
3
votes
1answer
121 views

Is it possible to have stratified train-test split of a set based on two columns?

Consider a dataframe that contains two columns, text and label. I can very easily create a stratified train-test split using ...
0
votes
3answers
43 views

Which algorithm to use for transactional data

I'm given a Dataset of transactions and asked to find insights for businesses. I'm extremely new to ML / Data science and have only been experiencing with KMeans. The dataset has the following ...
4
votes
1answer
42 views

Public datasets that show “cyclical” behavior

I am looking for any publicly available dataset that has a "cyclical" structure to it, in the sense that if I plot the data in a certain way, a loop becomes visible. A good example of this ...
0
votes
0answers
10 views

Does Call Detail Record(CDR) data record different cell-site/tower on incoming and outgoing call/text?

I am working on CDR data to find important locations of user. The data I have does not have coordinates so I will be using cell-site location as user's location. But after doing some working on data I ...
1
vote
0answers
8 views

Checkerboard artefacts vs distinct objects in GANs

I found a very good solution for getting rid of checkerboard artefacts in GANs: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/issues/190 Instead of using Transposed Convolution, use bilinear ...
0
votes
0answers
9 views

Free exchange rate data API for all countries at at least daily frequency since at least 2000?

I thought it would be a very common dataset: world exchange rates at daily frequency for every currency pair (or each currency's rate vs USD so triangulation can fill in for every other rate) But ...
0
votes
1answer
20 views

(Serious) Dataset of paedophilic Youtube comments (or similar)? [closed]

I think it would be useful to create a model that tries to predict whether a youtube comment is paedophilic - maybe the model should also take into account the channel name/description/front image. It'...
0
votes
0answers
10 views

Predicting a variably-placed value in a vector

I have $m$ vectors in $\mathbb{R}^n$, where $m >> n$, and I want to train a model to impute a value $x_i$ in $\mathbf{x}$, where $1 \leq i \leq n$ (and can vary by vector). For instance, I may ...
0
votes
1answer
10 views

Dataset to be used to learn NN regressions in classroom experiments

Can you suggest me datasets with intermediate size of about 1000-10000 to work with neural network regressions in classroom experiments? It seems that there are much more datasets for classification ...
2
votes
1answer
92 views

How do I read the cord_19_embeddings_2020-07-16.csv from the COVID-19 Open Research Dataset Challenge (CORD-19) on Kaggle?

What I'm trying to do I wanted to use the CORD19 word embeddings csv to map them to certain findings from the rest of the dataset, but as we can see there are no stings in the first column. The way I ...
2
votes
1answer
29 views

What does it mean that an hypotesis is consistent?

I am studying concept learning, and I am focusing on the concept of consistency for an hypotesis. Consider an Hypotesis $h$, I have understood that it is consistent with a training set $D$ iff $h(x)=c(...
2
votes
3answers
57 views

What is the best way for synthetic data generation while maintaining privacy?

For one of the projects where we are working as third party contractors, we need a way for the company to share some datasets which can be used for data science. It is not possible for the company to ...
0
votes
0answers
13 views

Should I filter my chess game-states?

I have written an AI which plays chess and learns from a dataset of game-states, where each state is a record of a board position and a corresponding result of the game. This causes some inequalities ...
1
vote
1answer
21 views

Cleaning a certain feature to predict salary using Machine Learning

Info: I am working on a dataset, and i would like to create a model that would predict salary. Columns are as follows: ...
1
vote
0answers
25 views

Will setting up time series data in this way cause data leakage?

I am trying to predict future stock market values using a gradient boosted tree model. As far as I know, gradient boosted trees use the data in one row, and only that row, to predict the target ...
0
votes
1answer
41 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
0
votes
0answers
11 views

Estimate class proportions of a feature, central limit theorem

haven't been feeling smart lately and this is probably the most trivial question ever but I really need to know. I'm trying to point estimate some population parameters. I sampled from 1000 randomly ...
2
votes
1answer
20 views

Missing at random vs missing not at random: What if it is both? (Does one imply the other?)

My understanding is that: Missing at random: Whether or not a variable's value is missing is dependent on the values of the other variables. Missing not at random: When the propensity for a variable'...
1
vote
0answers
36 views

Pandas how to change values in a column [closed]

I have data on mchip, chip schip and mcaid that contains the following columns/features: year (between 2000 and 2016), state, age and month. I have a problem with several years. For example, I want to ...
2
votes
0answers
9 views

Dataset suggestion: video dataset with accurate depth and pose info

Please suggest some datasets (can be synthetic, like rendered in blender or so) which have video frames (30fps preferred) with accurate ground truth depth maps and pose (rotation and translation for ...
0
votes
1answer
23 views

Exploratory Data Analysis on dataset divided by winners and losers

I have a dataset where I have features from winning tennis players and the other half are from a losing tennis players: ...
2
votes
1answer
44 views

List of numbers as classifier input

I am trying to use my data to predict the classes of the input. My data are as the following: ...
0
votes
1answer
24 views

How to determine sample rate of a time series dataset?

I have a dataset of magnetometer sensor readings which looks like: ...
1
vote
0answers
14 views

Datasets for software defined network (SDN)

I am working on a project with Software Defined Networks (SDN), and new to SDN area. I was checking around internet to find a benchmark dataset, which is a flow based dataset inorder to apply ML ...
0
votes
1answer
16 views

How to Transpose dataframe with custom output? [closed]

I have a data like this: I want to transpose it like this in python:
1
vote
2answers
42 views

Modality of data

Can anyone please explain in clear words what is generally meant by "modality of data"? I know what modality means with respect to distributions.
0
votes
0answers
15 views

Dynamic graph dataset

I'm working with dynamic graph embedding but I can not seek the dataset for my problem. Can you help me find dataset with some requires: A series of graph which represent for dynamic graphs. Each ...
1
vote
2answers
51 views

R : Counting the number of observations per category

I'm currently starting out in R and wondering how to count the number of observations per day, per node, per replicate from the below dataset, and store in a different data set. The original dataset ...
-1
votes
1answer
27 views

training accuracy greater than validation accuracy

The problem that I'm facing is that the training accuracy of my model is way higher than the validation accuracy, were talking about an approximate value of 0.2. ...
1
vote
0answers
17 views

How to deal with different audio formats for audio classification?

I am working on an audio classification problem statement to classify between two audio classes. I have collected samples from jotform, they are providing audio widget to collect .wav audio but it ...
2
votes
0answers
48 views

Where can i download a benign PE dataset? or at least which website is the best candidate for crawling and downloading normal executables?

I'm planning to gather a benign dataset for my ML malware detection model the problem I'm having is finding benign PE files, i just need a source that has a dataset of normal executables, i will scan ...
1
vote
2answers
21 views

Biasing SVM algorithm towards particular subset of data

I'm training an SVM model for sentiment analysis, based on social media data eg. tweets. The model will be trained using a small selection of a particular company's tweets in order to classify new ...
0
votes
0answers
14 views

RMSE in NGBoost example appears to be unreasonably high

I tried using Standford's ML Group NGBoost. I tried using this example provided by the authors of the repository. While training the output is along the lines of : ...
1
vote
0answers
24 views

What is important for Pharmaceutical companies to answer with Big Data Analysis?

I am a data scientist, and I have some biological background (genetics). I have been asked to give a talk for our customers from pharmaceutical industry. I should show them how they benefit from Big ...
-1
votes
1answer
22 views

How to Select Rows from Pandas DataFrame with condition [closed]

I have a dataframe (csv file) which consists of about 8000 songs (column name "Song")of e.g. 20 singers(column name "Artist"), so each singer has more than one song. I want to ...
1
vote
2answers
16 views

Negative Value in Covid-19 Dataset

I am currently practicing my data visualization skills with ourworldindata dataset. I noticed there is a negative value of new_cases in Benin country. Out of ...
1
vote
0answers
8 views

Orange 3 working with multiple datasets [closed]

I do not know if it is possible to somehow connect different datasets, that are related in some way, in Orange 3. I am working with EEG multiple files (of channel values results) and one main data ...

1
2
3 4 5
22