Questions tagged [data]
Questions mostly concerned with managing data, without focus on pre-processing or modelling.
0
votes
0answers
4 views
Writing slice by slice to a new .nii file in Python
I am using python 2.7.
I am writing code that reads from an initial nii file, modifies each slice of it, and then creates a new nii file and writes to it slice by slice. My main issues are I do not ...
0
votes
1answer
16 views
What algorithms are good to predict next numbers?
Let's consider we have several hundreds of numbers like ( 1, 2, 5, 8, 7, 15, 19, 8, 4, 6, ...) those are closed numbers of a stock on consecutive days for example. I like to know what algorithms are ...
0
votes
0answers
3 views
How do I create a universal customer ID within Google Cloud Platform? [on hold]
I had a few quick questions regarding the creation of a Universal Customer ID (UCID) within Google Cloud Platform (GCP). To make this easy, I'll start by providing some background, context, and ...
0
votes
0answers
10 views
How do I extract data out of cornell movie dialog corpus?
I am new in making chatbots.I want to make a chatbot for which I need a dataset.I decided to go with cornell movie dialog corpus. But I am unable to understand
the structure of their data . You can ...
0
votes
3answers
17 views
What best/correct algorithm/procedure to cluster a dataset with a lot 0's?
I'm new to statistics so sorry any major lack of knowledge in the topic, just doing a project for graduation.
I'm trying to cluster a Health dataset containing Diseases(3456) and Symptoms(25) ...
1
vote
0answers
24 views
Solving correlation and covariance with MySQL
I am new to data science taking my first course in Data analysis. I am trying to figure out where to start with a problem solving the correlation and covariance with my sql.
...
-1
votes
0answers
18 views
How to apply Kmeans clustering to find frequency of users
My data is in a csv file which contains user ids against their number of days since their last login. I have used the following code which i found in this website.
My main goal is to plot the users ...
0
votes
1answer
7 views
How to obtain and load a good initial data set for object localization?
I'm looking for a good data set for training a CNN based network to do object localization (i.e. a data set with class labels and bounding box data).
What is a good initial data set to use?
How can ...
0
votes
1answer
14 views
Replacing column values in pandas with specific column with multiple database operation?
Need to replace database of the column in specific refine query with multiple operations as mention in below image. Trying such operation as an individual, but can't understand which method to use can ...
0
votes
1answer
13 views
Do I need a datalake in my use case?
My web application stores usage data, for example:
tickets opened an closed
tasks executed
user scores
etc. I need to show dashboards and reports for usage and performance trends, like:
How many ...
-1
votes
0answers
11 views
Need road condition data for the Washington DC area [migrated]
I am taking a data science class and we are supposed to develop our own final project idea.
The general context of the idea I am considering is that there is a group that meets once per week. Based ...
0
votes
1answer
14 views
Weather data of Texas by premise zip
I need historic data of Texas' weather of the past ten years(2010-2018) by zip codes on monthly basis. is there an api or a website from where i can scrape the required data?
0
votes
0answers
16 views
Statistics of Model vs. Measured Data
I have 3 [150 x 60,000] arrays. One is measured data, the other 2 are models with inputs based on the metadata for the measured data array. In short, 150 altitudes at 60,000 specific date, time, and ...
1
vote
3answers
64 views
Data Cleaning without pandas [closed]
How can I clean a data csv file with the restriction of only using python and its standard library? No third party programmes such as pandas can be used. For example: removing a column from the ...
0
votes
0answers
10 views
input annotations quality check for large scale image data
while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use ...
0
votes
1answer
13 views
Software for automated database processing
I faced a problem which I'd like to solve w/o any programming. And looking for a software to do this.
I have a dataset, for example: (brand-id, brand-name, product-class-name;)
...
0
votes
1answer
39 views
Alternatives to doc2vec?
What are some alternatives to the doc2vec embedding model? I.e models that convert paragraphs/documents into vectors, not just models that take the mean/sum of the word embeddings of each word in the ...
0
votes
0answers
20 views
Data science short courses in Europe
Maybe my question is broad and even "bad", so excuse me for that!
I am looking for some good and intensive short courses in data science/machine learning etc. around Europe. By short i mean a week ...
0
votes
1answer
27 views
Missing Values In New Data
(Before someone marks this as duplicate - I'm not asking about training data, I'm asking about new data which has come in and needs to be classified)
Suppose I've got a dataset which has 5 predictors ...
0
votes
1answer
22 views
generate a report in R where the value in each Column is to be derived from different tables in sybase databasea
I work in a stress testing team. my boss has asked me to generate an interactive report every day using different tables in a Sybase database. I know the table structure and sql queries, but can't get ...
0
votes
1answer
33 views
Balance data using different criteria
I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID.
For each ID, I have a ...
1
vote
1answer
13 views
Different types of analysis which can be done with Webdata of users
I have rich customer data for site usage, particularly the web analytics data. Some of the data fields I have are:
Product Usage details
Geographic details
Session data: Capture what all they did ...
0
votes
1answer
9 views
Continuous values in a dataset
In a data analysis continuous data needs to be discretized. However in a computer it's impossible to have values $\in \mathbb{R}$, so what this mean ?
Thanks
0
votes
2answers
21 views
Is there a way to; easily, merge data into a CSV (or Excel) file without having to edit each file individually?
I have several marketing files that I need to download on a weekly basis.
Let’s say 15 different files. Each file contains the same field names: field 1, field 2, field 3, field 4, field 5, field 6, ...
0
votes
1answer
23 views
Dataset with features as SNOWMED codes and outputs as names of medications prescribed.What are some ML algorithms that suit this case?
So, i am constructing a dataset (from doctor prescription notes and diagnosis) where
X (input features) represent the various snowmed codes for illnesses/surgeries performed/chronic conditions etc.
...
0
votes
0answers
7 views
Does integrating data points that fall under a curve make sense?
I have data I've collected from the fanfiction website, Archive of Our Own. I want to know if there's a relationship between the length of a chapter and the average number of hits per chapter. I ...
0
votes
0answers
18 views
Folder Structure — Where to store data
Most of the python cookie-cutters that I've seen for data science (I've been looking at cookiecutter-data-science) and reading elsewhere, everything seems to indicate that its best practice to keep ...
0
votes
0answers
23 views
Multiple grouping in R [on hold]
I would like to group the parts of pipeline by the name (OZNAKA_ODS) and lenghth(Dolzina) of the pipeline.
Further i would like to write a new atribute table with only identical names of pipelineand ...
2
votes
2answers
184 views
Create a binary-classification dataset (python: sklearn.datasets.make_classification)
I would like to create a dataset, however I need a little help. The dataset is completely fictional - everything is something I just made up.
Since the dataset is for a school project, it should be ...
2
votes
0answers
46 views
Beginner Question Related To Data Science Course [closed]
I am Third Year B.Tech student from 3-tier college of India ,Here is no one fellow or collegous who has somebit knowledge about ML or Data science and I am purely sure that I have strong background of ...
7
votes
6answers
100 views
Is it advisable to combine two dataset?
I have two datasets on heart rate of subjects that were recorded in two different places (two different continent to be exact). The two research experiments aimed to find the subjects' emotions based ...
1
vote
1answer
26 views
Why not higher accuracy in Otto data?
On this site of Otto Group Product Classification Challenge, it is shown that best accuracy was possible with RandomForest method, but it was relatively low at 0.83. Accuracy with ANN and with Naive ...
0
votes
1answer
90 views
Validation-split of Keras fit function
Validation-split in Keras Sequential model fit function is documented as following on https://keras.io/models/sequential/ :
validation_split: Float between 0 and 1. Fraction of the training data
...
1
vote
1answer
16 views
How can I sort a data frame by groups? [closed]
I am using R. I have a data frame with "year , price, mileage" columns. I want to group the df by year first and then sort each group by mileage. How can I do this?
2
votes
1answer
29 views
Convert a csv file so that all attributes will become columns with corresponding element [closed]
so I have this dataset in a csv file. I would like to convert it into a matrix form.
The data currently looks like this:
...
0
votes
1answer
28 views
Image data augmentation with this function
Following function is from http://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#sphx-glr-auto-examples-neural-networks-plot-rbm-logistic-classification-py ...
0
votes
1answer
21 views
Handling imbalanced data by deleting over represented rows vs. adding under represented rows
I am currently working with a very imbalanced data set
(frauded credit card data from kaggle, which has 492 rows of frauded cards and over 280,000 rows of non-frauded cards).
As much as I know, there ...
1
vote
0answers
27 views
Transposing a dataframe (time-series)
I'm working on event-forecasting problem for a supermarket, and the data frame I have looks like (sorry for formatting):
...
3
votes
2answers
32 views
How to generate data if algo itself is involved in the process with a feedback loop?
I have an algorithm which would be a rather easy classification task with a set of features and a class output which I would like to solve with a machine learning algo.
But I am having issues and ...
0
votes
0answers
28 views
Releasing data with datajoint
I want to release some data along with a publication. The data are part of a long processing chain stored in MySQL and processed with datajoint. What is the best way to release the data such that the ...
2
votes
1answer
28 views
Algortihm for making predictions from minimal data
I am working with classification problem.
I have a dataset with a lot of features. A lot of them can easily determine class. On production, I want to ask the user to provide me with only part of the ...
3
votes
0answers
110 views
Interactive labeling/annotating of time series data
I have a data set of time series data. I'm looking for an annotation (or labeling) tool to visualize it and to be able to interactively add labels on it, in order to get annotated data that I can use ...
1
vote
0answers
15 views
Having the target and data row - how to find the correct model
I am coming from a web developer background, and having no data science background.
I am looking to skill-up and get some answers for a current task: find a model that returns a predicted value same ...
0
votes
0answers
8 views
Approach for dealing with missing/non-existent data
Not sure if this is the right place.
I have a set of match results from an international, 32-team competition, stretching back to 1991 (20,000 matches). Each year, there are different teams competing....
4
votes
4answers
490 views
How to know for sure if we can learn from a given data or not?
I want to know that given a set of data and a target, how we can know for sure whether we can learn from that data to make any inference or not?
0
votes
0answers
21 views
Data Science : from handson person to manager
I am a data science hands on person with some experience in SAS/R/python/excel and exposure to techniques like regression, time series, clustering, decision tress etc to name a few. Now I have been ...
1
vote
1answer
27 views
Prepare data : many products per row
I would like to find how to change my data structure to make it compatible with a ML model, with the following constraints :
A Product is defined by {id, price, continuousVariable1, ..., ...
1
vote
1answer
22 views
Is there an API to extract questions and labels from yahoo answers?
I want to work on a ML project which involves the family and relationship category from yahoo answers. I want to extract questions from yahoo answers (only the title) with the label as the category it ...
1
vote
0answers
17 views
Time Series Autocorrelation Estimation
How do I find joint PDF f(xi,xj) needed for estimating autocorrelation from 1-D time series data X, using the kernel windowing technique?
I am well aware of kernel windowing technique for estimation ...
4
votes
3answers
98 views
Understanding data normalisation
So I know that when we have different parameters with different value ranges we have to standardise these values. Also, I read that when a parameter does in fact require higher values then we should ...
