Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0 votes
1 answer
12 views

Is it a good idea to add predictable features in dataset before training?

For example, let's say a dataset has 100+ columns. And the ml has to recognize a complex pattern containing 1000 different conditions. And one of those conditions is, the integer from the first column ...
user avatar
  • 123
0 votes
1 answer
27 views

Feature Engineering on transactional dataset clustering

I have a data set with transactions details from different business (roughly 1 thousand business entities). Each row is a transaction. The structure of the dataset is as follows: client_id Sex Age ...
user avatar
0 votes
0 answers
10 views

How do I train an instance segmentation model with an image classification dataset?

Say I have a clean dataset of different types of utensils. By clean, I mean the images are already cropped, augmented, and ready to use for image classification models. If I want to detect multiple ...
user avatar
  • 11
0 votes
1 answer
14 views

Str.contains and isin function do not return all correct rows of dataframe

Given a list of strings L1 L1 = ['a', 'b', 'c'], I need to extract the rows which contain the values given in list L1. I used the isin function: ...
user avatar
  • 23
0 votes
0 answers
9 views

Where can I download ImageNet-22k

I only found ImageNet-21K on the ImageNet website, but I need ImageNet-22K. Where can I download ImageNet-22k?
user avatar
0 votes
0 answers
18 views

How Can I find the medical image datasets for congenital heart defect

I am working on implementing the image segmentation model for diagnosing a patient with a congenital heart defect. But I couldn't find the image datasets. please, anyone, help me to Find the Image ...
user avatar
0 votes
2 answers
31 views

If you are making a ensemble model does training data on base models have to be different from one another

I was reading this article talking about ensemble models. I was interested in the max voting model using 3 base learners. However, I am a little confused about the process. Currently, I'm thinking it ...
user avatar
  • 13
2 votes
1 answer
17 views

What Preprocessing is Needed for Semantic Search Using Pre-trained Hugging Face Transformers?

I am building a project for my bachelor thesis and am wondering how to prepare my raw data. The goal is to program some kind of semantic search for job postings. My data set consists of stored web ...
user avatar
0 votes
1 answer
23 views

Topic modelling or Keyword extraction for a small dataset

I am working on a project where I have a dataset which contains very less data. These are the comments of people. I have only 130 lines with 10 words per line. My goal is to identify the common topics ...
user avatar
0 votes
0 answers
26 views

How to give label to synthetic data

I am doing some research. I have created a dataset with 4 features using a uniform distribution. I think it is a right way to do it but I am not sure. I have done sum the features values/numbers of ...
user avatar
0 votes
0 answers
18 views

Generating data

I have been stuck in a Problem, for more than 2 weeks now. I need to generate data set, that have actual Probability. I am doing an experiment, my ML framework gives me probability intervals for the ...
user avatar
0 votes
1 answer
14 views

Python library for recommendation engine

I have a dataset containing records of books issued to students in college. It comprises the following fields: Department of student Date on which book is issued to the student Details of the book ...
user avatar
  • 1
0 votes
0 answers
12 views

Keras changing from ndarray to tensorflow data leads to error

I'm training a model with tensorflow keras and numpy input with: ...
user avatar
  • 101
0 votes
0 answers
9 views

Describe changes in atmosphere data - mathematical solutions

I interpolated isotopic data (2H or 18O) from 1970 and plotted it on a map. I did the same with data from 2010. The interpolated data are gridded data which have changed over time. Up to now I have ...
user avatar
  • 101
0 votes
0 answers
22 views

Best way to store "small" data

I have a small dataset consisting of around 200 measurements of different light sources. The light is measured with a broadband and an IR diode, and the resulting voltage is sampled it with 4 ...
user avatar
  • 1
0 votes
0 answers
8 views

How to classify a "Trip" from multiple GPS pings of each Unique ID?

I have a huge file of lat/long & timestamp data from vehicles and I need to identify "Trips" from these multiple pings of Each vehicle ID. Trips have a "Start" & "end&...
user avatar
0 votes
0 answers
7 views

Good ratio of positive and negative examples for siamese network

I'm currently training a siamese network on textual data. While optimising my model I came across the question what is the best ratio of positive and negative examples. Let's say I try to identify the ...
user avatar
  • 101
1 vote
3 answers
43 views

Huge datasets for practise

I am currently doing a project which requires a dataset. May I know whether there are any free large public datasets that has the following characteristics: greater than 3GB non-tabular (each txt ...
user avatar
0 votes
0 answers
11 views

Selecting Right data and amount of data

Multiple Data Sources are needed to make Credit or Fraud Decision on (loan/credit card) applications. But each data pull costs money to buy. So how do we optimize a) Number of data sources to pull ...
user avatar
0 votes
0 answers
25 views

How we describe scientifically easy and difficult dataset?

Sometimes we hear that a dataset is easy or difficult for a same task (like classifying cats and dogs). How we can describe it in scientific way( math or probability or statistics). For example we ...
user avatar
0 votes
1 answer
10 views

Twitter data scraping: API vs Advabced Search

Need to analyze tweets on a specific topic, my application (for using Twitter API) was not approved. Instead, I tried to do it manually using Twitter Advanced Search. However it's more burdensome than ...
user avatar
  • 111
0 votes
1 answer
16 views

Sampling from earths landmass

Given a sphere which resembles earth, I want to sample points where land would be. I am struggling to find a dataset to sample from, and even to find a dataset which I could use to generate a set of ...
user avatar
  • 3
0 votes
0 answers
22 views

How to calculate dataset and feature sparsity/density

I have a dataset with 8 features and 30,000 samples but which is probably a sparse sampling. I would like to quantify how sparse or dense the dataset and individual features are, as described in the ...
user avatar
  • 101
0 votes
0 answers
19 views

Problem with images captured for dataset creation?

I am trying to create a custom dataset using the camera installed to my car dashboard. And I tried to train my model to find the potholes in the road. But unfortunately, due to the reflection of the ...
user avatar
0 votes
0 answers
11 views

Dealing with survey data across several years

I am working on a dataset and a major part of this dataset is the survey results in the past 5 years. The survey data includes 8 different perspectives such as self-rating, satisfaction rate, etc. All ...
user avatar
  • 1
1 vote
1 answer
17 views

Which format is preferrable to publish book dataset (plain or preprocessed)?

When I decide to publish collection of book texts as a dataset, should I do some preprocessing first or should I publish "plain texts"? For example, https://huggingface.co/datasets/...
user avatar
  • 113
0 votes
1 answer
23 views

Why weak supervision works?

I've learned that weak supervision combines multiple labeling methods to generate labels for a large dataset. I can't understand why generated labels can be used to train a more accurate model than ...
user avatar
  • 3
5 votes
4 answers
383 views

Calculating Potential Usefulness of Acquiring Additional Data [closed]

Imagine Anne has a labeled training dataset for a machine learning prediction problem. There is an opportunity to acquire more data from an agent, at a cost. However, before she decides to acquire ...
user avatar
  • 59
0 votes
1 answer
16 views

Data Mining of unresearched data for a master's degree final project

So, I have to start thinking about the topic of my final project in a data science master's degree (business oriented, although I can choose any unrelated field) and one of the requirements is to ...
user avatar
2 votes
1 answer
23 views

All classification models except neural network giving 100% accuracy

I have a dataset of size (140,000, 10) containing 1 dependent variable. I used MinMax scaler on independent variables. For the target value, there is a class imbalance of 94% 0's and 6% 1's. Used ...
user avatar
  • 23
0 votes
0 answers
8 views

Changing values with topographical data!

I am currently interpolating data from various measuring stations, after which this data is available as gridded data. In a further step I want to adjust the data according to a topographical data set,...
user avatar
  • 101
1 vote
0 answers
14 views

How can I combine (or not) multiple datasets for training a model

I am trying trying to predict if a customer is likely to churn or not. I have multiple datasets, from multiple different customers, of which some have churned and some that have not. I am trying to ...
user avatar
  • 11
0 votes
2 answers
50 views

Joining on columns with duplicate values - clean before merging or after merging?

I am joining on two data sets on a column which has duplicated values in both datasets. Is it better practice to remove the duplicates and make the values I am joining on a primary key in both ...
user avatar
0 votes
0 answers
9 views

combining two dataset

My question is simple, is there any condition or methodology to combine two different datasets? In a nutshell, I have two different studies with different variables that I would like to combine in ...
user avatar
0 votes
2 answers
15 views

Can I change my data split after making a model?

I created a model but realized I incorrectly split the data between training and test sets so my prediction accuracy is low. Is it unethical to redo my modelling attempt over from the start and ...
user avatar
0 votes
0 answers
14 views

How to merge two datasets with different spatial granularity?

Goal: create a dataset from data with different spatial granularity to explore the relationship between socio-demographic variables and intensity of mobile use. Data: CDR MobilePhoneData: city area ...
user avatar
0 votes
0 answers
9 views

Real world data anonymity trade-off problem

Background: We are a small business based in France who advise the private cosmetic sector across the country. We have a good relationship with our clients and have set out on a new project whereby ...
user avatar
  • 101
1 vote
0 answers
35 views

Should I train multiple models or single model with respective data for multiple parameters?

In one of my projects I have 7 output classes. However, while this makes up a baseline set of results, I also want to test the impact of some experimental parameters. To avoid any information ...
user avatar
  • 111
0 votes
0 answers
10 views

Shapiro Wilk test for normality for 6 treatments in R

I am trying to learn R and am struggling with some homework for an agriculture module. I need to check for normality in a dataset on specific growth rate, and I can very easily check for normality ...
user avatar
0 votes
0 answers
13 views

Is this XGBoost model tending to overfit?

Here is the list of hyperparameters that I used: ...
user avatar
0 votes
0 answers
8 views

Organizing datasets, dataset version control, MLOps and other questions

I am currently looking into structuring data and work flows for my ML end to end pipeline. I therefore have multiple problems, and ideally I am looking for one platform that can do all: Visualize and ...
user avatar
  • 1
0 votes
0 answers
5 views

Is the Dataset XiangyaDerm available anywhere?

I've searched far and wide, does anybody know how to access the XiangyaDerm dataset? They say in their paper, that it is accessible. It has 150k images of skin lesions, which is way more than ...
user avatar
  • 131
0 votes
1 answer
28 views

convert time series data set to supervised for deep learining

I have dataset like so I want to use that for prediction of time series with deep learning. I have this function to make it supervised ...
user avatar
0 votes
0 answers
19 views

seasonality in classification model

I am building a classification model to predict customer status a year from a given time. There seems to be some seasonality, for example, more changes occur in Summer than in Winter etc. so my ...
user avatar
0 votes
0 answers
20 views

Train an LSTM on separate sequences of different lengths

My case is the following: I want to train a sequential classifier to recognize what action is being performed given sensors observations.My data consists of 10 executions of an assembling task for 10 ...
user avatar
  • 131
0 votes
2 answers
33 views

How to evaluate data imputation techniques

I have a data set with 29 features 8 if them have missing values. I've tried Sklearn simple imputer and all it's strategies KNN imputer and several Number of K Iterative imputer and all combinations ...
user avatar
0 votes
0 answers
9 views

Models / General guidelines in joining two datasets via timestamp

So I have two datasets (dataset1, dataset2) that are related in this way: (Think of a checkout counter in a supermarket) A customer comes in with an item. The cashier registers the item to dataset1 ...
user avatar
0 votes
0 answers
9 views

Where should I find electrolytic capacitor ageing data

I am trying to get a dataset of Electrolytic capacitors ageing and I am not being able to find one that shows the ripple current and the voltage in order to calculate its Equivalent Series Resistance (...
user avatar
0 votes
1 answer
25 views

How to set the same number of datapoints in the different ranges in correlation chart

I am beginner in working with machine learning. I would like to ask a question that How could I set the same number of datapoints in the different ranges in correlation chart? Or any techniques for ...
user avatar
0 votes
0 answers
38 views

Song playlist recommendation system

I want to build a recommender system to suggest similar songs to continue a playlist (similar to what Spotify does by recommending similar songs at the end of a playlist). I want to build two models: ...
user avatar
  • 1

1
2 3 4 5
28