Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dataset]

A dataset is a collection of data, generally represented in tabular form, with columns signifying different variables and rows signify different members of the set.

1
vote
0answers
9 views

Need help with confusing dataset formats for Images and annotations

I have a more general question, which I could not answer after quite a bit of google. Actually, I got more confused. I hope you can share some light. I am familiar with deep learning, Keras but come ...
0
votes
2answers
33 views

Loops in R programming,

I want to update this Remaining column in this table below: How do I do this in R programming? I tried while (n.Remaining>0) { n.remaining <- n.total-n.expense} Desired Output:
0
votes
1answer
24 views

Choosing sample from large dataset?

How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
0
votes
0answers
10 views

Spatio-temporal Dataset - Machine Learning [on hold]

Which ML algorithm to use when dealing with spatio-temporal dataset. Also, can anyone direct me to a tutorial to learn how to perform such ML process? Regards.
0
votes
1answer
15 views

Can you apply PCA to part of your dataset?

I am working with kaggle dataset that has over 130 features composed of 116 categorical and 14 continuous features. I plotted the heatmap for the 14 continuous variables and found that most of them ...
1
vote
0answers
24 views

Are there any public datasets about mental health of patients containing physiological and psychological symptoms?

I would like to segment mental illnesses with clustering using machine learning. To do so I need training dataset which contains physiological and psychological symptoms of an subject. The closest ...
0
votes
0answers
23 views

Filtering unique row values in SQL, Advanced

My data looks like this: Number(String), Number2(String), TransactionType(String), Cost(Integer) For number 1, Cost 10 and -10 cancel out so the remaining cost is 100 For number 2, Cost 50 and -...
0
votes
0answers
7 views

Split timeline for training LSTM network

I have faced some trouble in splitting my dataset before feeding the data into an LSTM network. My data are time series, so that the only feasible way is to split them into training and testing part ...
0
votes
1answer
16 views

Train, test and submission files - what am I supposed to do with all of them?

this might be very beginner's question. I'm working on Kaggle's HomeCredit Default Risk problem which has among others dataset train, test and submission files as can be seen in the link provided. The ...
1
vote
1answer
36 views

What kind of data visualization should I use?

I'm going to program a customized phone keyboard where some letters are larger than others, depending on how often I misstype them. For example, if I often pressed "w" instead of "e", I'd make the "e" ...
2
votes
1answer
21 views

What is the best way to organize the datasets for my task?

I am working on a task — geolocation estimation of Twitter users by using tweets only. I collected tweets (and users) from more than 6000 people in Twitter. Each user is associated with a city. In ...
0
votes
0answers
24 views

How to do this? Substituting a general word with a random particular? [closed]

For a pet project I need general words substituted with a random particular. Example, if I write [country] [color], it could give out: Argentina Green or India Blue. Or anything random from the set of ...
0
votes
2answers
38 views

Dataset Merging

I have several datasets (All looks like this). The Problem is, that the same user is on several datasets. I need to merge the different sets in a way, that if the username is the same, the frequencies ...
0
votes
0answers
29 views

Working with Stanford twitter datasets in R

I intend to use frequency data and sentiment analysis on the Stanford twitter data. I have a couple issues though. One is the size of the data, which I want to reduce, but how do i do this? A ...
0
votes
0answers
22 views

Mixed linear Model

I have a dataset with G1 (Grade of first semester)and G2(Grade of second semester)and G3 the final grade. I need to model it using linear mixed model with repeated measure to study the variation ...
0
votes
0answers
17 views

unused argument error in R

Hi I'm running this code on R studio: datasetrmna <- read_xlsx('datasetrmna', header = TRUE, sep = ';') and I get an error saying unused argument. ...
1
vote
1answer
39 views

Feature name extraction directly from dataset

I would like to visualize my features using the code below. However, I am getting an error that my features are being recognized as "nan" instead of their actual names. Instead of the below ...
0
votes
0answers
9 views

Alternative source/way to download MIMIC-III

Is there an alternative source/way to download MIMIC-III (Medical Information Mart for Intensive Care) database (complete or partial) besides Physio Net ? For example, PhysioNet Challenge program ...
1
vote
1answer
21 views

Dataset where svm performance is significantly different from random forest

Is there a specific dataset where svm performs significantly better or worse than random forest? I know that the performance could depend on the dataset but is there a specific dataset?
0
votes
0answers
8 views

Where can I find IQ data?

I was trying to find IQ data by year and by region. For the analysis, I want not just the average IQ for each year and region but the distribution of IQ. I am particularly interested in higher moments,...
0
votes
0answers
15 views

Machine learning approach in Irrigation

I am new to machine learning. I am working on a project of Machine learning irrigation problem. I need to study on particular crop (ex. Rice crop). I have to apply the machine learning approach to ...
0
votes
1answer
8 views

Is there any data set containing list of windows software and the category each one of them belongs to?

I wanted a dataset which lists many windows softwares according to their categories for a project. eg: anaconda: software development, roblox: gaming, VLC: video player etc. I searched but could not ...
0
votes
0answers
15 views

(help please)Data analysis in a health sector data

I have got about 20000 number of data where it has columns like Age, Sex, Address and final diagnosis. Age is from a day to a 90 year. I am trying to use Ml or analysis but confused. Can anyone help ...
2
votes
1answer
22 views

features importance

Suppose I have dataset labeled with two classes such as healthy and unhealthy and I applied feature selection (features importance)on dataset. How can I know that features are important to which ...
0
votes
0answers
7 views

Why is pre and post silence important when collecting speech data?

I've been wondering why it's important, when collecting speech data for AI machine learning, to collect pre and post silence for speech data? Is it just so the machine learns and understands the ...
1
vote
1answer
27 views

Issues with training SSD on own dataset

I'm new to ML and trying to train a SSD300, with some Keras-Code github.com/pierluigiferrari/ssd_keras I found on github. For training I'm using an own (very small) dataset of objects that are not in ...
2
votes
1answer
18 views

Reinforcement Learning on real time data over a web server

Question: is it possible to implement a reinforcement learning model over a NodeJS server? This server would be receiving binary forms of data (open /close; yes/no) in real time. The objective for ...
0
votes
0answers
10 views

Advice on advanced analysis for end term Project

I have an end of term project for a class in Applied Data Analytics. I am a software engineer of 18 years and I have learned a lot so far using R with little problems. This question is less ...
0
votes
0answers
13 views

Read images into RAM in chunks

I have a large dataset of images in a directory that I wouldn't be able to read into RAM all at once. Is there a way for me to load say 1000 images at a time without listing out all the images in the ...
0
votes
2answers
35 views

amount of data required for a good ann mdel

I am new to deep learning and started with ANN. I have a dataset with 15 parameters and 2100 rows. r2_score with Multi linear regression and random forest models is around 85% but when i try an ANN ...
0
votes
0answers
12 views

For those who use Dask in a single machine, what is your hardware specifications?

I want to use dask for a big dataset, if considering 2 000 000 lines is big. I did try and that is kinda slow for me. What is sure is I do not master the library so it is possible to miss something in ...
1
vote
1answer
13 views

Census dataset alternatives

I'm looking to replace some of my code samples which use the Census dataset. Criticism is that there is some bias and ML fairness issues around this popular dataset. What other public datasets for ...
1
vote
1answer
67 views

Best CNN architecture for binary classification of small images with a massive dataset [closed]

The title has it all... Any tip is welcomed. Should I use a very deep convolutional neural network ? Use a large amount of filters ? Parallel layers ? Dataset examples: 1) "Good" 2) "Bad"
1
vote
1answer
59 views

Kmean clustering on text data

I have a large raw dataset on crime and I want to cluster the data using k-mean, However, I get this Error when I enter this code ...
1
vote
1answer
20 views

Dataset Made of Multiple Datasets

I recall a tweet by Andrej Karpathy sometime a while ago about a ML dataset made up of cifar-10, mnist, ImageNet, and maybe others. I want to use that dataset, but I can't find it. Does anyone know if ...
1
vote
2answers
29 views

Loading own train data and labels in dataloader using pytorch?

I have x_data and labels separately. How can I combine and load them in the model using torch.utils.data.DataLoader? I have a dataset that I created and the ...
0
votes
0answers
21 views

How to combine scipy sparse csr matrix to pandas dataframe. | Combining text feature with numerical features

I have both textual and numerical features on my data set. (1 textual feature, 4 numeric features). For the textual feature I use tfidf vectoriser to convert my textual data into numbers. X_train is ...
1
vote
1answer
13 views

Converting VOC2012 annotation .xml to segmentation .png

Does anyone know a good way to convert VOC2012 annotations files in xml to SegmentationClass ...
1
vote
2answers
54 views

Is it ok to train the model only on the interested part of the data?

Let's say I have a dataset where one feature is 'Car type' : say 'A', 'B' and 'C'. The test set consists of samples where 'Car type' is always equal to 'A'. Therefore, should I train my model only on ...
1
vote
0answers
16 views

is there any package in python similar to tidyrin R? Except Pandas [closed]

is there any package in python similar to tidyr in R? Except Pandas
0
votes
0answers
5 views

How can I find a large dataset of medical images for cancer classification [migrated]

I am looking for a large image dataset >20K to be used for cancer classification algorithm. Where can I look given that all public available datasets are maximum 1K in size which is much less than ...
0
votes
0answers
9 views

QA-Datasets for non-englisch languages?

I wonder if they are communities like for german where you can access german data sets for NLP task. For english there is the QA-dataset Squad. I wonder if such thing exists for german too?
1
vote
0answers
9 views

mlbench synthetic datasets for python

Is there any implementation of the synthetic datasets from mlbench in python? I am looking for these implementations in particular. I've found sklearn.datasets for synthetic examples, but they do not ...
0
votes
0answers
18 views

Project structure - many projects share same large dataset

I have a bunch of projects for my job that are largely unrelated except they use the same data, which is pretty big on disk in csv format. I want these to exist separately from each other and I ...
1
vote
0answers
9 views

Need inputs for Food data set to predict consumer ordering behaviour [closed]

I am looking for food order data set that shows the consumer list and type of food they order regularly. Like : customer_number, gender, day, time, choice_of_food, food_category, quantity(grams), ...
2
votes
1answer
21 views

Normally distribute occurence or counts

I am creating a mock of sales data. One of the columns is salesperson_id where each id can occur more than once (a salesperson can have multiple sales). I want to ...
2
votes
0answers
15 views

Sub-sampling so that sample statistics match population statistics

I want to investigate the impact of various testing strategies on a product. Let's say chairs. I start with 500 random chairs that I've picked up from garage/yard sales. They come in all shapes and ...
3
votes
1answer
21 views

Group datetime64 values per week in dataframe

I want to plot a bar of my data grouped per week. I made the following dataframe, with nonsense data, but it should do the trick. What I want to do is plot per week a bar chart (%) Does anyone know ...
1
vote
4answers
38 views

Which statistical model should I run to get a good model

I'm working on an analysis to analyze projected man-hours vs actual man-hours used for different teams. My data looks like this below. The software I am using is SAS. I've tried a HP linear ...
3
votes
0answers
32 views

How to filter this signal to get a heartbeat signal?

I am giving my first steps in data analysis, gathering/cleaning. To learn, I am trying to create a simple code that can detect heartbeats from color variations from the image coming from the camera ...