Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
1answer
40 views

VlookUp: how to loop it? [closed]

I am not an experienced user of Excel because I usually use python but I am obliged to use it this time. This is the big picture of my case. The problem is: I have a sheet with a column with some ...
0
votes
0answers
13 views

How to identify whether or not my dataset is sufficient enough to be learnt from any model at all?

I created my own bipartite dataset where 1 group consists of disease and another group consists of genes. The number of diseases is significantly lower that number of genes. Furthermore, one disease ...
0
votes
0answers
9 views

What statistical test would I use to test that both data sources are matching

Suppose I am only working with two datasets, Data Source 1 with columns Customer ID and ...
1
vote
0answers
27 views

How to classify product by specific category without machine learning?

I am working on a product classification problem which I have to identify product category. Say for the category, there are 5 levels (Big / medium / small / detail / double detail) 5 million ...
0
votes
0answers
21 views

How do I export my data into .h5 format?

I am trying to make my data useful for to use in this model, i.e for Hierarchical Novelty Detection for Visual Object Recognition. I need to prepare my own dataset in a format like this: ...
1
vote
1answer
66 views

How to split a dataset into train and test sets for time series (multiple step-multiple output forecasting)?

I am trying to use an LSTM neural net to do multiple step / multiple output forecasting (I predict multiple values in one time knowing some values in the past). But, I have realized that I must be ...
0
votes
0answers
18 views

How to do numpy matmul broadcasting between two numpy tensors?

I have the Pauli matrices which are (2x2) and complex ...
1
vote
0answers
26 views

Predicting electricity consumption in a household

Please I need your help in brainstorming on a problem. I have a community, in this community, every household has a 'power device' that powers the household. Households need to buy power units from a ...
0
votes
0answers
9 views

Which are the techniques and methods to find date-time observation level for data set?

I have a dataset of five years data with timestamps for each row. I want to analyze it at different levels, for example: by hour of the day, day of the year, day of the week, day of the month, month, ...
2
votes
2answers
72 views

Is it correct to join training and validation set before inferring on test-set?

I would like to know if is a correct procedure to join training-set and validation-set together, in order to train the model on ...
0
votes
0answers
17 views

What insights can be gathered from electric/gas smart meters within the field of machine learning or deep learning?

I'm currently doing my graduate research for my Master's degree at a particular firm. Initially, I was supposed to classify images of smart meters in homes using deep learning technology. ...
1
vote
1answer
36 views

Technical term for using regular expressions to classify text?

Background I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext. He wants to classify the claims as either having been "approved", "denied", ...
1
vote
1answer
113 views

Combining heterogeneous data sets for more powerful machine learning

Suppose we have two data sets of movie reviews; one from IMDB and one from Rotten Tomatoes (RT). Each entry has a written-review and a score attached to it. The concatenated datasets might look like ...
1
vote
0answers
29 views

Problem On Class Imbalanced Data

I am getting an F-Score of 0.99 on the train_test_split data, but only getting 0.40 for a competition's test data. I am oversampling with random forest (after ...
4
votes
3answers
223 views

Where can I find a social network image dataset?

I am supervising a programming project whose goal is to detect offensive images on social networks. I would like to have a representative dataset of social network images. It would be best if the ...
0
votes
0answers
53 views

what will be the best algorithm for find similar users based on user features , betwenn knn, euclidian or jaccard?

I have user data table containing user data like university, gender, city, id etc. I also have a matrix data-set like this . [
0
votes
0answers
17 views

How to generate artificial datasets in MATLAB based on behavior of real datasets?

To understand: Let's consider a scenario. I have real datasets of 2 buildings showing different behaviors of occupants. Now to expand the analysis for multiple buildings, I want to generate simulated ...
0
votes
0answers
9 views

How to combine multiple data sources where subjects are unknown and there is no ID attribute to match subjects?

I have two data sets. Data set A: data about lifestyles, health conditions and SOCIOECONOMIC and Data set B: data about travel behavior and also SOCIOECONOMIC. We don't know whether they are the same ...
0
votes
1answer
401 views

Creating new column in dataframe based on conditions in 2 other columns [closed]

I would like to create a new column in my dataframe based on values from both the gender and experimental_grouping columns. As I have it written below, the column ...
0
votes
1answer
32 views

Dataset of Tweets by Same User

I was reading the Keras documentation on shared layers, and used the example of a model built to classify whether 2 tweets were from the same user or not. Is there any dataset that I could use to do ...
0
votes
2answers
48 views

Convert non-binary value into binary value?

I'm kind of a newbie in ML I have a dataset like: ...
1
vote
0answers
20 views

Split Dataset in two Explainable Subsets

is there a general approach for splitting a dataset into two or more subsets so that a classification problem can be solved with higher accuracy for the resulting separate subsets? I have a very ...
0
votes
2answers
97 views

Would K-means be Appropriate to Use with Four or More Variables?

Just a general question that I'm trying to mentally visualize. I'm fairly new to using k-means clustering and have used it before on two variables, which creates a 2-D plot of points. I also know, ...
0
votes
0answers
28 views

Creating artificial blurred image to train neural network for super resolution

I would like to create blurred images from images obtained from large pdf with the intention of creating a training data set. I would like to create the blur as shown in the uploaded picture. Which ...
0
votes
0answers
83 views

While reading the pages of a PDF file using Python, I get the following error. There are 300 pages in pdf file

CalledProcessError: Command '['java', '-Dfile.encoding=UTF8', '-jar', 'C:\Users\105051884\AppData\Local\Continuum\anaconda3\lib\site- packages\tabula\tabula-1.0....
2
votes
2answers
601 views

Looking for animals dataset for deep learning classification

Do you know any datasets that contain animals and their accurate classifications? I am looking for any dataset that categorizes animals. For example: a dataset with insects, with an image of an ...
2
votes
1answer
218 views

Smart data split (train/eval) for Object Detection

I am looking for a smart way of splitting object detection data (images with labelled objects inside them) while taking into account the distribution of the objects themselves and not just the images. ...
0
votes
1answer
57 views

How to approach the PlantVillage dataset?

I'm working on the PlantVillage dataset and i want to predict the type of the disease from the image of a leaf. The dataset is labeled in pairs (Type of the plant,Healthy/name of the disease). I'm ...
1
vote
2answers
40 views

Removing # from header

I have a $(418,2)$ matrix and I want to convert it to csv. so I write: np.savetxt('titanic1.csv', Sol, fmt='%.2f', delimiter=",",header="PassengerId,Survived") ...
0
votes
0answers
18 views

How can solve data imbalance issue in MATLAB for a PatternNet?

I have built a Pattern recognition NN in MATLAB. I have two datasets, one for training and one for testing each of 4500 samples. The number of classes is 3. Out of the 3 classes, class #1 has approx ...
1
vote
2answers
33 views

Does it make sense to concatenate datasets to improve accuracy of model?

I'm planning on training an NER model, I already do have a large corpus but I did find one more large corpus and I'm quite confident that I can source even more corpora and format its data to my needs....
0
votes
2answers
98 views

Classification Dataset which is linearly non separable

I am trying to find a dataset which is linearly non-separable. I checked the Iris dataset and the UCI website says: The data set contains 3 classes of 50 instances each, where each class refers to ...
0
votes
0answers
9 views

Algorithm for automatic annotation with landmarks

I worked on iris detection in real-time using deep learning, so it so hard to get an annotated dataset for iris detection with landmarks, and manually annotation takes a long time and this work should ...
1
vote
1answer
54 views

If the entire production dataset is known is advance, should I use it when fitting normalization parameters?

I found in multiple sources the recommendation not to fit the normalization parameters on the combined train/test dataset when evaluating the model, to prevent data leakage. I presume this ...
0
votes
1answer
40 views

How to choose our data set wisely?

I have a couple of questions and I was wondering if you could answer them. I have a bunch of images of the cars (side view only). I would like to train a model with those images. My objects of ...
0
votes
0answers
19 views

Feature vector of linear model

I read this paper that applies logistic regression to a dataset generated from a simulation they created. The dataset contains a set of binary vectors (called challenges) that looks like this: ...
1
vote
1answer
157 views

How to treat a categorical variable which can have multiple values in regression?

I am working on a regression problem and has a categorical feature 'CATEGORY' which can take as many as 1600 categories. On top of that, each observation can have any number of categories in that ...
3
votes
1answer
53 views

How to predict based on multiple samples?

I am relatively new to ML so I apologies in advance if my question shows lack of understating of the field. The problem A particular study course has a high drop-out rate and we want to reduce it. ...
0
votes
0answers
20 views

How can you automate collecting curriculum vitae data?

I'm doing a machine learning project, for which I need data from thousands of curriculum vitae. For this, I need to collect data from the employees of some 50 specific companies. From each company, I ...
0
votes
1answer
13 views

how to export the data edited to a new excel file

The provided code imports the excel file and removes the not NaN values and store it in df2. Now, How do i save the changes to an another file. ...
0
votes
1answer
112 views

How to delete a row if a values in a column is not NaN

In the given code it displays only the rows without NaN values but i want only the rows with NaN values in a provided column everything has to be removed ...
1
vote
2answers
157 views

Regarding imbalanced multilabel dataset

I am doing multilabel news classification in python language.The dataset I have has two files. First CSV contains articles at each row. Second CSV contains corresponding labels to each article. Here ...
0
votes
1answer
16 views

Machine learning model using keywords for binary decision

I have a some experience from Uni with convolutional NN and edge detection, but haven't much explored the other types of machine learning models. I was wondering if there might be one that is suited ...
2
votes
1answer
30 views

how to balance the data set with different number of observations

I am pretty new to ds field and recently I was working on a self project of the clustering model. My goal is to create clusters and see in each cluster what the common features are between customers. ...
-2
votes
2answers
39 views

What is Data Science and Machine Learning and what language mostly used to program? [closed]

I am newbie in Data Science, Machine Learning and any related to data science but I want to try it. Unfortunately, googling makes it tedious and complicated so I hope to be answered by anybody who's ...
0
votes
0answers
16 views

Co-training independent views

I found a tutorial on co-training that mentions that I need two views for training, which are conditionally independent. Could someone explain to me what this means regarding datasets?
2
votes
2answers
138 views

World's Wine's Dataset

I'm conducting a machine learning project and therefore looking for a world's wine's dataset to create a classification model based on user's ratings. Can someone recommend me any datasets? Haven't ...
1
vote
1answer
49 views

Creating an Object Detection model with images and coordinates of bounding boxes

I am trying to build a custom Object Detection model which can detect guns from a given image. The dataset is from here. The dataset has a good number of images and each image has 4 coordinates of ...
0
votes
0answers
63 views

Metrics parameter in model.compile function in Keras for imbalanced data set

Accuracy for an imbalanced data set is not relevant and therefore I use precision and recall to evaluate my model. However, whenever I train a model in Keras a metrics parameter must be specified in ...
0
votes
0answers
69 views

An epoch step is taking too much time on GPU!

I am trying to train a CNN on phoneme recognition. My dataset is composed up of 50 Million sample, each sample has 120 feature (signal parameters). My input's shape is (11,120) and the target is a ...