Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data]

Questions mostly concerned with managing data, without focus on pre-processing or modelling.

0
votes
0answers
4 views

Writing slice by slice to a new .nii file in Python

I am using python 2.7. I am writing code that reads from an initial nii file, modifies each slice of it, and then creates a new nii file and writes to it slice by slice. My main issues are I do not ...
0
votes
1answer
16 views

What algorithms are good to predict next numbers?

Let's consider we have several hundreds of numbers like ( 1, 2, 5, 8, 7, 15, 19, 8, 4, 6, ...) those are closed numbers of a stock on consecutive days for example. I like to know what algorithms are ...
0
votes
0answers
3 views

How do I create a universal customer ID within Google Cloud Platform? [on hold]

I had a few quick questions regarding the creation of a Universal Customer ID (UCID) within Google Cloud Platform (GCP). To make this easy, I'll start by providing some background, context, and ...
0
votes
0answers
10 views

How do I extract data out of cornell movie dialog corpus?

I am new in making chatbots.I want to make a chatbot for which I need a dataset.I decided to go with cornell movie dialog corpus. But I am unable to understand the structure of their data . You can ...
0
votes
3answers
17 views

What best/correct algorithm/procedure to cluster a dataset with a lot 0's?

I'm new to statistics so sorry any major lack of knowledge in the topic, just doing a project for graduation. I'm trying to cluster a Health dataset containing Diseases(3456) and Symptoms(25) ...
1
vote
0answers
24 views

Solving correlation and covariance with MySQL

I am new to data science taking my first course in Data analysis. I am trying to figure out where to start with a problem solving the correlation and covariance with my sql. ...
-1
votes
0answers
18 views

How to apply Kmeans clustering to find frequency of users

My data is in a csv file which contains user ids against their number of days since their last login. I have used the following code which i found in this website. My main goal is to plot the users ...
0
votes
1answer
7 views

How to obtain and load a good initial data set for object localization?

I'm looking for a good data set for training a CNN based network to do object localization (i.e. a data set with class labels and bounding box data). What is a good initial data set to use? How can ...
0
votes
1answer
14 views

Replacing column values in pandas with specific column with multiple database operation?

Need to replace database of the column in specific refine query with multiple operations as mention in below image. Trying such operation as an individual, but can't understand which method to use can ...
0
votes
1answer
13 views

Do I need a datalake in my use case?

My web application stores usage data, for example: tickets opened an closed tasks executed user scores etc. I need to show dashboards and reports for usage and performance trends, like: How many ...
-1
votes
0answers
11 views

Need road condition data for the Washington DC area [migrated]

I am taking a data science class and we are supposed to develop our own final project idea. The general context of the idea I am considering is that there is a group that meets once per week. Based ...
0
votes
1answer
14 views

Weather data of Texas by premise zip

I need historic data of Texas' weather of the past ten years(2010-2018) by zip codes on monthly basis. is there an api or a website from where i can scrape the required data?
0
votes
0answers
16 views

Statistics of Model vs. Measured Data

I have 3 [150 x 60,000] arrays. One is measured data, the other 2 are models with inputs based on the metadata for the measured data array. In short, 150 altitudes at 60,000 specific date, time, and ...
1
vote
3answers
64 views

Data Cleaning without pandas [closed]

How can I clean a data csv file with the restriction of only using python and its standard library? No third party programmes such as pandas can be used. For example: removing a column from the ...
0
votes
0answers
10 views

input annotations quality check for large scale image data

while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use ...
0
votes
1answer
13 views

Software for automated database processing

I faced a problem which I'd like to solve w/o any programming. And looking for a software to do this. I have a dataset, for example: (brand-id, brand-name, product-class-name;) ...
0
votes
1answer
39 views

Alternatives to doc2vec?

What are some alternatives to the doc2vec embedding model? I.e models that convert paragraphs/documents into vectors, not just models that take the mean/sum of the word embeddings of each word in the ...
0
votes
0answers
20 views

Data science short courses in Europe

Maybe my question is broad and even "bad", so excuse me for that! I am looking for some good and intensive short courses in data science/machine learning etc. around Europe. By short i mean a week ...
0
votes
1answer
27 views

Missing Values In New Data

(Before someone marks this as duplicate - I'm not asking about training data, I'm asking about new data which has come in and needs to be classified) Suppose I've got a dataset which has 5 predictors ...
0
votes
1answer
22 views

generate a report in R where the value in each Column is to be derived from different tables in sybase databasea

I work in a stress testing team. my boss has asked me to generate an interactive report every day using different tables in a Sybase database. I know the table structure and sql queries, but can't get ...
0
votes
1answer
33 views

Balance data using different criteria

I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID. For each ID, I have a ...
1
vote
1answer
13 views

Different types of analysis which can be done with Webdata of users

I have rich customer data for site usage, particularly the web analytics data. Some of the data fields I have are: Product Usage details Geographic details Session data: Capture what all they did ...
0
votes
1answer
9 views

Continuous values in a dataset

In a data analysis continuous data needs to be discretized. However in a computer it's impossible to have values $\in \mathbb{R}$, so what this mean ? Thanks
0
votes
2answers
21 views

Is there a way to; easily, merge data into a CSV (or Excel) file without having to edit each file individually?

I have several marketing files that I need to download on a weekly basis. Let’s say 15 different files. Each file contains the same field names: field 1, field 2, field 3, field 4, field 5, field 6, ...
0
votes
1answer
23 views

Dataset with features as SNOWMED codes and outputs as names of medications prescribed.What are some ML algorithms that suit this case?

So, i am constructing a dataset (from doctor prescription notes and diagnosis) where X (input features) represent the various snowmed codes for illnesses/surgeries performed/chronic conditions etc. ...
0
votes
0answers
7 views

Does integrating data points that fall under a curve make sense?

I have data I've collected from the fanfiction website, Archive of Our Own. I want to know if there's a relationship between the length of a chapter and the average number of hits per chapter. I ...
0
votes
0answers
18 views

Folder Structure — Where to store data

Most of the python cookie-cutters that I've seen for data science (I've been looking at cookiecutter-data-science) and reading elsewhere, everything seems to indicate that its best practice to keep ...
0
votes
0answers
23 views

Multiple grouping in R [on hold]

I would like to group the parts of pipeline by the name (OZNAKA_ODS) and lenghth(Dolzina) of the pipeline. Further i would like to write a new atribute table with only identical names of pipelineand ...
2
votes
2answers
184 views

Create a binary-classification dataset (python: sklearn.datasets.make_classification)

I would like to create a dataset, however I need a little help. The dataset is completely fictional - everything is something I just made up. Since the dataset is for a school project, it should be ...
2
votes
0answers
46 views

Beginner Question Related To Data Science Course [closed]

I am Third Year B.Tech student from 3-tier college of India ,Here is no one fellow or collegous who has somebit knowledge about ML or Data science and I am purely sure that I have strong background of ...
7
votes
6answers
100 views

Is it advisable to combine two dataset?

I have two datasets on heart rate of subjects that were recorded in two different places (two different continent to be exact). The two research experiments aimed to find the subjects' emotions based ...
1
vote
1answer
26 views

Why not higher accuracy in Otto data?

On this site of Otto Group Product Classification Challenge, it is shown that best accuracy was possible with RandomForest method, but it was relatively low at 0.83. Accuracy with ANN and with Naive ...
0
votes
1answer
90 views

Validation-split of Keras fit function

Validation-split in Keras Sequential model fit function is documented as following on https://keras.io/models/sequential/ : validation_split: Float between 0 and 1. Fraction of the training data ...
1
vote
1answer
16 views

How can I sort a data frame by groups? [closed]

I am using R. I have a data frame with "year , price, mileage" columns. I want to group the df by year first and then sort each group by mileage. How can I do this?
2
votes
1answer
29 views

Convert a csv file so that all attributes will become columns with corresponding element [closed]

so I have this dataset in a csv file. I would like to convert it into a matrix form. The data currently looks like this: ...
0
votes
1answer
28 views

Image data augmentation with this function

Following function is from http://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#sphx-glr-auto-examples-neural-networks-plot-rbm-logistic-classification-py ...
0
votes
1answer
21 views

Handling imbalanced data by deleting over represented rows vs. adding under represented rows

I am currently working with a very imbalanced data set (frauded credit card data from kaggle, which has 492 rows of frauded cards and over 280,000 rows of non-frauded cards). As much as I know, there ...
1
vote
0answers
27 views

Transposing a dataframe (time-series)

I'm working on event-forecasting problem for a supermarket, and the data frame I have looks like (sorry for formatting): ...
3
votes
2answers
32 views

How to generate data if algo itself is involved in the process with a feedback loop?

I have an algorithm which would be a rather easy classification task with a set of features and a class output which I would like to solve with a machine learning algo. But I am having issues and ...
0
votes
0answers
28 views

Releasing data with datajoint

I want to release some data along with a publication. The data are part of a long processing chain stored in MySQL and processed with datajoint. What is the best way to release the data such that the ...
2
votes
1answer
28 views

Algortihm for making predictions from minimal data

I am working with classification problem. I have a dataset with a lot of features. A lot of them can easily determine class. On production, I want to ask the user to provide me with only part of the ...
3
votes
0answers
110 views

Interactive labeling/annotating of time series data

I have a data set of time series data. I'm looking for an annotation (or labeling) tool to visualize it and to be able to interactively add labels on it, in order to get annotated data that I can use ...
1
vote
0answers
15 views

Having the target and data row - how to find the correct model

I am coming from a web developer background, and having no data science background. I am looking to skill-up and get some answers for a current task: find a model that returns a predicted value same ...
0
votes
0answers
8 views

Approach for dealing with missing/non-existent data

Not sure if this is the right place. I have a set of match results from an international, 32-team competition, stretching back to 1991 (20,000 matches). Each year, there are different teams competing....
4
votes
4answers
490 views

How to know for sure if we can learn from a given data or not?

I want to know that given a set of data and a target, how we can know for sure whether we can learn from that data to make any inference or not?
0
votes
0answers
21 views

Data Science : from handson person to manager

I am a data science hands on person with some experience in SAS/R/python/excel and exposure to techniques like regression, time series, clustering, decision tress etc to name a few. Now I have been ...
1
vote
1answer
27 views

Prepare data : many products per row

I would like to find how to change my data structure to make it compatible with a ML model, with the following constraints : A Product is defined by {id, price, continuousVariable1, ..., ...
1
vote
1answer
22 views

Is there an API to extract questions and labels from yahoo answers?

I want to work on a ML project which involves the family and relationship category from yahoo answers. I want to extract questions from yahoo answers (only the title) with the label as the category it ...
1
vote
0answers
17 views

Time Series Autocorrelation Estimation

How do I find joint PDF f(xi,xj) needed for estimating autocorrelation from 1-D time series data X, using the kernel windowing technique? I am well aware of kernel windowing technique for estimation ...
4
votes
3answers
98 views

Understanding data normalisation

So I know that when we have different parameters with different value ranges we have to standardise these values. Also, I read that when a parameter does in fact require higher values then we should ...