Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Why kind of data is appropriate for prediction and what kind is not?

What relationship between data is appropriate for predictive modeling and what type is not? How do we ensure we select the appropriate type of data for predictive modeling and not use data that is not ...
HelloDarkWorld's user avatar
0 votes
1 answer
14 views

How to create a multi label text classification model for small dataset in production

I have a multi-label text classification dataset which is very small around 80Kb, I am only going to receive a small amount of data for training from my client. But it is expected to build a high ...
Aayesha Qureshi's user avatar
0 votes
2 answers
34 views

Difference between (None, 2000, 8) and (2000, 8)

Input: a tensorflow dataset with 17000 items: ...
Fabi's user avatar
  • 11
0 votes
0 answers
18 views

Differentiating between runs that start at different timestamps

I have a CSV file of the latitude and longitude coordinates of trains in the format [timestamp, runID, latitude, longitude]. The problem is that I have tracked ...
make nice rhombus's user avatar
0 votes
0 answers
5 views

Which dataset could be a good choice to train Environment Sound Classification model for user environment awareness while wearing earbus?

Which dataset could be a good choice to train an Environment Sound Classification model for the following use case: use the model in the earbuds/earphones to detect important sound events in the user'...
Danijel's user avatar
  • 173
0 votes
0 answers
10 views

Dataset with replicates: aggregation or not?

I am currently developing a neural network tailored to a regression problem using a synthetic dataset derived from an experimental simulation campaign. Given the stochastic nature of both the ...
Andrea Ferrari's user avatar
0 votes
1 answer
38 views

Outdated Transformers TextDataset class drops last block when text overlaps. Replace by datasets Dataset class as input of Trainer train_dataset?

Why I try to replace the transformers TextDataset class with datasets Dataset class I stumbled upon this when I tried to make ...
questionto42's user avatar
0 votes
0 answers
13 views

How to identiyfy issues in structure or missing values in netCDF-file that contains weather data?

during processing of weather data files (wind x,y 10m), i get the follwing error: height3' not present in all datasets and coords='different'. Either add 'height3' to datasets where it is missing or ...
otk's user avatar
  • 101
0 votes
0 answers
14 views

How to generate a netCDF weather forecast for a missing day based on yesterdays forecast?

as can be seen here met.no, the forcast is missing for 20210823. This forcast for each day is a 66 hour forecast, mening it will cover the next "missing" day. Not knowing much about netCDF ...
otk's user avatar
  • 101
0 votes
0 answers
15 views

Iterating more quickly, shorter feedback cycles, general workflow practices

(caveat: I've not written that much ML code yet) When I'm writing non-ML code I like to create short feedback loops to check the validity of my code. I use unit-tests or sometimes short helper tools (...
Niels Bom's user avatar
  • 101
0 votes
1 answer
26 views

Where can I get 5000+ classified images of zoo animals? [closed]

please help! We are college students doing this for a project. The project is using neural networks and want to build a model that takes in an input of a colored image of an animal and outputs the ...
user90061's user avatar
0 votes
1 answer
27 views

Binary Classification Dataset Distribution

I have a question regarding the distribution of my dataset. I have a binary classification task which I need to label. I now want to know how I should distribute the labels for optimal training / ...
Apatus's user avatar
  • 1
0 votes
0 answers
6 views

Estimating the Cost of Creating an Image Dataset like GQA and Relevant Sources

I am currently working on a project that requires an extensive image dataset, similar to the GQA dataset. I am in the early stages of planning and would like to get an estimate of the cost involved in ...
Patrick Amadeus Irawan's user avatar
0 votes
1 answer
23 views

What are these 'code' columns in this csv and what does Pandas pivot method do with it?

In a csv discussed in the book HoML (hands on machine learning), what do the values in capital letters represent? Are these some sort of code? Also, it seems the pandas.pivot() methods does something ...
Yorick Dantuma's user avatar
0 votes
0 answers
28 views

How can I identify coverage types in NFL games using Computer vision

I am currently working on a project that classifies coverage types from sports highlights using advanced computer vision techniques. Next Gen Stats effectively utilizes tracking data to identify ...
Shah Zeb's user avatar
1 vote
1 answer
51 views

Correct way to take a subset of a dataset?

I am attempting a binary classification problem (using Weka). My dataset has 100,000 rows, 14 attributes (1 output variable). It takes already too long just to open the dataset in excel so I just know ...
FlexMcMurphy's user avatar
0 votes
0 answers
18 views

Should I log transform the data if the dataset is small?

Suppose I have a dataset with only about 25 rows. All the relevant columns are all integers. The ranges of them go from 1 million to 100 million, and the distributions are all skewed right. In this ...
DSGuest's user avatar
0 votes
0 answers
9 views

How to make duplicates unique in a DataFrame robustly?

I have the following table. I was able to parse it and display it as a DataFrame with pandas in python I need to change the Ref for every value that is duplicate such that it becomes unique. For ...
bem22's user avatar
  • 101
0 votes
1 answer
21 views

How should I plot 2D data as 3D plot?

I got data (1503, 38697) (cell(datapoint), genes(features)), each elements has integer values (gene expression). so something like ...
containletters's user avatar
0 votes
0 answers
30 views

How can I read hdf5 files stored as 1-D array. and view them as images?

I have a large image classification dataset stored in the format .hdf5. The dataset has the labels and the images stored in the ...
Aleph's user avatar
  • 111
0 votes
0 answers
45 views

Looking for datasets on automobile parts information for machine learning

I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
Anand 's user avatar
0 votes
0 answers
7 views

How to get / calculate statistics about voxceleb2?

VoxCeleb2 contains ~1M utterances from ~6000 speakers (multilingual). I want to get statistics about voxceleb2: number of utterances & speakers for each language I tried to find the metadata ...
user3668129's user avatar
0 votes
2 answers
60 views

Seeking datasets for training a Language Model on U.S. mortgage loan processes

I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
Anand 's user avatar
0 votes
0 answers
29 views

propper feature encoding

I working with the following data set also here is it's detailed description of "packet_dat" column I can't understand how I can encode packet_dat column into proper feature so my ...
Roma's user avatar
  • 101
0 votes
0 answers
12 views

Packet data needed fot the model

I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification Here is packet structure from the README file. Assuming I ...
Roma's user avatar
  • 101
0 votes
0 answers
52 views

Proper way to build time series dataset

I have daily time series but need to predict weekly total. Here is example: date col0 col1 col2 col3 col4 target 0 2023-01-01 00:00:00 -1.021 0.708 -0.564 -0.502 0.244 -106.538 1 2023-01-02 00:00:...
aplusk's user avatar
  • 1
0 votes
0 answers
15 views

Feature selection on datasets with both categorical and numerical features

I'm proposing a novel methodology for feature selection in the context of tabular datasets that contain both numerical and categorical features. In order to prove the efficacy of my methodology, I ...
Francesco De Santis's user avatar
0 votes
0 answers
23 views

statistical analysis of two data sets have different number of sampling

I have two datasets from my statistical fault analysis: one with 400 iterations and another with 500 iterations. Both datasets have the same maximum and minimum values, and some data points are ...
Fadi's user avatar
  • 1
0 votes
0 answers
24 views

How to make a dataset for Audio Speech Recognition of Arabic Language

I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
Sijran's user avatar
  • 1
0 votes
0 answers
32 views

Open Source Model or dataset for attention map visualization

Helloo (: I'm trying to develop a python application where users can upload an image e.g. of their website and receive a heatmap or attention map in return. This map should indicate where users are ...
Vic Wild's user avatar
0 votes
0 answers
14 views

What do I need to take into account if I want to test one model on different test datasets?

I'm just new to machine learning, I want to test a generative model on two different test datasets to compare its performance on different datasets. What do I need to take into account about the ...
Lynnnn's user avatar
  • 1
0 votes
0 answers
24 views

Model for small sample time series

Im a total novice and i need to estimate some kind of relation-proof model(Granger test results, correl matrix are already provide some evidence) with following dataset: 20 observations (2001-2021), 4 ...
Maqar Mocha's user avatar
0 votes
0 answers
30 views

Optimal ML classification approach

Background: I have an app data (impressions, user activities) that I can use as features for a multiclass classifier (5 classes). I just want to discuss about some things that our team is having a ...
Marc Atanante's user avatar
0 votes
1 answer
24 views

how to build model using two input dataset in which there is no common column to merge or combine

I want to create model for truck company in which trucks delivers the car for customers.i have two data sets. one is customer details like how many cars they want from particular area or terminal and ...
prema's user avatar
  • 1
0 votes
1 answer
46 views

Very basic but how to understand data statistically for machine learning?

I’m trying to solidify my statistics so I really know how to use them in my analysis/models. However my concept of statistical testing gets completely messed up by context. I’m unsure defining exactly ...
donutmonster's user avatar
0 votes
1 answer
38 views

Time Series Forecasting for Multiple Store Sales with Simultaneous Timestamps

I have a sales dataset with each store having a unique identifier. The dataset contains daily sales data for each store over a period of around two-years. I'm looking to build a time series ...
SIDDHARTH CHAVAN's user avatar
0 votes
0 answers
7 views

Systematic way of selecting internet texts for a machine translation corpus / dataset?

I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
tschomacker's user avatar
0 votes
0 answers
12 views

Seeking Guidance on Creating Sample Data for AI Business Use Case

Subject: Seeking Guidance on Creating Sample Data for AI Business Use Case Dear StackExhange Family I am currently working on an AI project aimed at providing AI embaded solutions to HealthCare, ...
TARUN KUMAR's user avatar
1 vote
0 answers
29 views

Determining the start and stop of vehicle movement by GPS

There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
Abraham Lincoln's user avatar
0 votes
0 answers
28 views

For Finetuning Llama 2, what form of data is required?

I am working on a customer service chatbot project. I have files of different product's manuals on which i need to train the LLM which is Llama 2. As these are manuals so they not in Q/A form. In this ...
divertiss Blogger's user avatar
0 votes
0 answers
19 views

pdf dataset - personal information publicly available for unredaction project

I'm working on a research project, which is focused on demonstrate ineffectiveness of document redaction methods. I'm looking for a dataset of pdf documents which contains generalities about real ...
goemon07's user avatar
0 votes
0 answers
40 views

divide a large group of people into subgroups based on two parameter

note in advance: I'm new to data-analysis and although my major civil engineering taught me about statistics, I did not apply it the way I would have encountered in real life, or in this field of work....
Abdulrahman Sheikho's user avatar
0 votes
1 answer
28 views

Binary Classification of Images- CNN

I am learning ML and am working on a CNN problem where I need to classify images of CATS and DOGS. The way I have setup the labels is that cats are 1 and dogs are 0. I have made the final output layer ...
Hussain Bhavnagarwala's user avatar
0 votes
0 answers
17 views

RMSE of whole and part of test dataset

Can anybody help me to understand the behavior of metrics (RMSE namely) when testing model? I have NN with 1 hidden layer for regression task. RMSE equal 0.07 for external test dataset. But if I break ...
CUB_REV's user avatar
0 votes
0 answers
8 views

for multi-tasking training should I use big datasets as one file or each table as one csv?

Assuming we want to create a dataset of books. There would be dataset A: id_book, title, Author, Subject, Topic, Body [paragraph 1,2,3], Sites ... I was wondering what if i split each book into one ...
Magos Xiaomi's user avatar
0 votes
0 answers
42 views

No answers on gpt.h2o.ai if change to "personal MyData Collection" on the platform

I have downloaded pre-trained models BERT, llama-2 and some others (for CPU). I wanted to test, on the h2ogpt platform what output would each provide. My problem is how the knowledge of the models ...
Magos Xiaomi's user avatar
0 votes
0 answers
7 views

how to convert these string columns so that i can put the data to the regressor

https://drive.google.com/file/d/1yQR1xJvzWd2UGgGQS2bHhdywZEE_Nu0J/view?usp=sharing I have to change the name of the columns: city, batting_team, bowling_team, batsman, non_striker, bowler. These ...
gaurav najpande's user avatar
1 vote
1 answer
70 views

Alternative Source for Downloading OTB Dataset for Visual Object Tracking

I'm interested in downloading the OTB dataset for visual object tracking. The official site (http://cvlab.hanyang.ac.kr/tracker_benchmark/index.html) appears to be down. Does anyone know of ...
Reza Azami's user avatar
1 vote
1 answer
49 views

A good set of datasets/models for testing an NLP technique

I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
Anonymous's user avatar
0 votes
0 answers
25 views

How can I perform an analysis of the NHIS imputed income variables?

I have downloaded five family income variables from https://nhis.ipums.org/nhis-action/variables/group?id=economic_income (INCPPOINT1, INCPPOINT2, INCPPOINT3, INCPPOINT4, INCPPOINT5) for they years ...
tryingtogetsmth's user avatar

1
2 3 4 5
30