Questions tagged [data]

Questions mostly concerned with managing data, without focus on pre-processing or modelling.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

How can I export sprint report data from Spring Jira repository for research purposes?

Basically, my requirement is to export all data related to a sprint in a particular project in an opensource Jira repository, https://jira.spring.io/ Ex: The sprint report for Spring XD as follows. I ...
-1
votes
0answers
7 views

What are the technologies necessary to build a Car surface detection system for dents and scratches?

I am fairly new to Machine Learning and Data Science. I have been assigned a project in which I have to build a system that collects images of a car from 4 angles using different cameras and then ...
0
votes
1answer
15 views

Exploratory Data Analysis on dataset divided by winners and losers

I have a dataset where I have features from winning tennis players and the other half are from a losing tennis players: ...
0
votes
0answers
10 views

How can a CNN account for spectro-temporal constraints in neural data?

What are there the best ways to leverage the unique "geometrical" constraints of spectro-temporal signal representations (architecture, filter shapes, data augmentation, etc.)? For example, ...
2
votes
1answer
138 views

How can I count the number of matching zero elements between two numpy arrays?

I have a function that returns the predicted accuracy of a time-series model. I have two equally-sized numpy arrays, one for the actual direction and one for the predicted direction. I'm classifying ...
1
vote
2answers
18 views

Modality of data

Can anyone please explain in clear words what is generally meant by "modality of data"? I know what modality means with respect to distributions.
0
votes
0answers
11 views

How do I efficiently load data from disk during training of deep learning models in pytorch?

I'm trying to train a deep learning model without loading the entire dataset into memory. My main question is, what's the best way of doing this? It seems like HDF5 is a common method that people ...
2
votes
4answers
29 views

Scikit-learn: train/test split to include have same representation of two different types of values in a column

I have a dataset of online purchase orders that contains two types of customers: Customers who have an account and thus are known customers with unique customer number. Customers who do not have an ...
1
vote
0answers
13 views

pandas: reshaping the dataset, moving column name as rows [closed]

I currently have the following table: ...
-1
votes
2answers
24 views

Keep column after groupby in pandas [closed]

I have a data set like following : the dataset has multiple values of search date and i wish to filter rows where ['orig','dest','dep_date','ret_date','airline'] have the lowest price along with the ...
0
votes
1answer
12 views

Historical exchange rate data via World Bank API?

I can see a massive list of indicators available via the World Bank API here (may take 1-2 minutes to load) I found exchange rates for various currencies into USD, but cannot see how to get exchange ...
0
votes
0answers
17 views

Compare two columns and visualize third column based on comparison in Power BI

I am new to Power Bi, learning myself and I got a dataset with some questions. Most of the questions are about comparing two columns and visualize the effect on the third column. For example , lets ...
-2
votes
0answers
36 views

Multi linear regression, python, Issue with 2 unique values instead of heterogeneous values [closed]

Good day, I have thought several days before posting that question, and I really need help now : I am doing a Machine Learning / Data Science project with Python And I have done a simple linear ...
1
vote
1answer
31 views

Dealing with missing data

I have a question about data cleaning. I am a novice and have just started learning in this field so please pardon my ignorance. Suppose there are two columns and based on some samples taken from both ...
0
votes
1answer
19 views

The OLAP (On-Line Analytical Processing) cube with 4 dimensions [closed]

A typical OLAP cube looks like this: As I can see, this cube can work with 2 or 3 dimensions, but what if I have 4 dimensions to produce facts? Should I use star schema instead when having more than ...
0
votes
0answers
15 views

Determining common columns among many datasets without having to manually inspect them

I have a bunch of datasets (10+) and I want to be able to join/ relate them to one another through columns that may contain similar values. For example, date and location are common columns that could ...
-2
votes
1answer
18 views

Accuracy of a forecasting model for prediction of COVID-19 occurence

My goal is to find the best performing forecasting model for the occurrence of COVID-19 in Toronto. I pre-train the network with data on the occurrence of SARS in ten countries and Toronto. Then I ...
2
votes
0answers
37 views

Regression dataset with categorical features

I have thought of a regression technique that I want to try on several datasets. I would like these datasets to have the following properties: Be a tabular dataset (no images). Have at least 20k rows,...
0
votes
1answer
8 views

RANKING TUMORS FROM MOST MALIGNANT TO MOST BENIGN

I have a classification problem classifying tumors as benign or malignant. However, I want to go a step further to provide a ranking of these tumors as most malignant to most benign. Are there any ...
-1
votes
0answers
11 views

Theoretical Neural Network weighting of features for time series data?

Theoretical question Say I have time series data and expectations at all times and it's features/properties that are defined by 20 different variables. There are different regimes that the data goes ...
-2
votes
0answers
14 views

How to map certain values to a category?

I'm new to Tableau and I have a question regarding mapping values. Say I have the categories Country and Region for an employee. All employees have a Country value, but not a Region value. For ...
1
vote
1answer
21 views

Random Forest in R with only character variables

I am new to using random forest in R an my goal is to identify the independent variables which have the highest impact on the dependent variables. I am looking at sales data, and sales is my dependent ...
-1
votes
0answers
14 views

Suggestion needed concerning feature engineering for continuous and categorical variables

During feature engineering for regression problem, shall we apply separate techniques for numerical variables and categorical variables OR there is any technique that can be applied in a combined ...
-1
votes
2answers
28 views

Data Selection & Visualization Tool

I am looking for a tool that will show neatly all the rows from a large table (100GB+) for complex selection conditions from an sql/text/csv table. DB may be on linux or windows. Looking for easy ...
0
votes
0answers
14 views

Why use raw.githubusercontent.com link in a script?

In the book "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" The author provides the link: DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml2/master/" My ...
0
votes
1answer
23 views

Higher variance in PCA can mean, that data structure is less informative?

I have the possibility to describe data with two different data structures, both data structures are some sort of approximation to the true data. I would like to compare the two data structures with ...
0
votes
0answers
11 views

Normalization around the nominal value while avoiding to get too big results

Lets say we measured a signal N times. Each measurement contains T time instance. For example for N=4 measurements and T=5 time instances the data matrix is as follows: ...
0
votes
1answer
22 views

How to adjust/smooth a certain number using constants or rules

Hi, I am handling a dataset with a customer purchase history. The field ord_cnt represents the purchase without coupon usage, and cpn_ord_cnt represents the purchase with coupon usage. There are two ...
2
votes
2answers
39 views

Data Imbalance in Regression Tasks [closed]

Having read a lot about class imbalance in classification tasks, I'd like to know what is the methodology for data imbalance in regression tasks. Particularly, - What is the procedure to check for ...
3
votes
2answers
186 views

Is it acceptable not to transform() test data after train data is being fit_transform()-ed

We know that the best practice in data preprocessing (such as standardization, Normalization, ... etc) is that while we perform fit_trasform() on the training data, ...
1
vote
0answers
27 views

List of undesirable faces?

An Australian sporting league simulated live crowds using cardboard cut-outs to fill empty stadiums during COVID-19. Fans were allowed to submit their own faces to be used on the cutouts, which ...
0
votes
0answers
18 views

What's the best way to store and then call 3 Million+ records for training?

Good afternoon all, I am working on a Deep Learning project where I'm proceeding in batches (rather than a continuous pipeline). Essentially I've built a function that can convert raw data into ...
1
vote
3answers
52 views

Continuous VS Categorical variable

In the dataset I have a continuous variable AGE and categorical variable AGE_CATEGORY as well. They are both highly correlated. ...
1
vote
2answers
15 views

Tools for nutritional value tracking and visualization

I want to track the nutritional value of different flours over time. What I got so far is a data table like this: ...
0
votes
0answers
9 views

Resources for building data pipelines

I am finding it difficult to find sources which walk you through building a data pipeline from scratch, for example from: - SQL Server (where I want to do my cleaning/ETL) - R/Python (where I want to ...
0
votes
1answer
82 views

Data augmentation for multiple output heads in Keras

I have a transfer learning based two output classification problem. So, accordingly, I have formatted my data to have X_train as a ...
0
votes
1answer
12 views

Suggestion of dataset

I am implementing my own deep network, but I am not so good at calculus so my network only works for binary data in the moment. I have been searching for big tabular datasets that are for binary ...
0
votes
0answers
23 views

Comparing the two feature sets

I am working on a classification of two feature sets derived from a dataset. We first obtain two feature matrices derived from two feature extraction methods. Now, I need to compare them. However, the ...
0
votes
1answer
10 views

Getting the earliest date (duplicates due to several call ids and agents)

I am trying to get the earliest call interaction, with only the agent that interacted at that call and the customer id. Using the following code, I still get duplicates: ...
1
vote
2answers
36 views

What's my target variable?

I am beginner in data science. I have this "aids" dataset from "mdhglm" package in R. dataset = aids, info = Repatead Measures on AIDS Data ...
0
votes
0answers
15 views

Using Google Trends data for comparability across states/regions

I am interested in using Google Trends data in order to analyse some searches across say U.S. States. Say I wanted to compare search intensity for the keyword "Donald Trump" but across different ...
0
votes
1answer
22 views

Vertical lines of overlapping points in Seaborn Swarmplot

When I plot my data to see its distribution via Swarmplot, these two vertical lines that seems to even overlap occur in the figure. It seems to occur even around values that are not that dense, such ...
0
votes
0answers
7 views

tensorflow feature columns

I am a beginner in TensorFlow. So while going through an article I came across tf.feature_columns being used. I understand the general use of it,but what I am confused about something.There were some ...
2
votes
1answer
22 views

When training NN, how do data loaders work on large datasets?

How do you normally organize large data-sets for easy loading when training Neural Networks? I have a largeish data-set which cannot fit into memory, it consists of 200,000 samples, with 10k samples ...
1
vote
1answer
38 views

Logistic Regression: Is it viable to use data that is outdated?

TLDR: Want to predict who makes the playoffs (1,0), but there are more playoff spots now than there were in the past, is it okay to use that past data? I want to use binary logistic regression on MLB ...
0
votes
1answer
44 views

What's an appropriate datastore for variable length sequence data for PyTorch consumption?

I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats. I need a datastore that can rapidly ...
0
votes
2answers
22 views

CNN training on Different Datasets

I currently want to train a CNN but I have two small datasets that are slightly different because of the camera setup that captured the images. I'm interested in ultimately tuning the neural network ...
1
vote
0answers
12 views

Tableau: Dealing with Date values [closed]

Im a noob at tableau. I have a quick question I have data that doesn't explicitly list the date. (There is no column labelled "date" ) However there are columns with the following headers: "Income ...
0
votes
0answers
27 views

Machine learning model for record linkage of unstructured data

I have a list of string. Each string has position based distinct information. On the other hand, I have a totally different structure data ( say xml ). How do I match the string and the XML based on ...
0
votes
1answer
27 views

Building a content-based music recommendation system

I am trying to build a recommendation system in Python that recommends songs based on a playlist. What I have is two datasets: 1. One dataset consists of 350 songs from my playlist and 13 acoustic ...

1
2 3 4 5
12