Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [python]

Python is a general-purpose, dynamic, strongly typed language with many 3rd-party libraries for data science applications.

0
votes
0answers
2 views

How to split a Bunch type

I have a dataset of images saved in python, in a Bunch type. In this buch, one key is 'train' that assume the values 1 or 0. I need to split them in X_train and X_test in a deterministic way (in ...
0
votes
0answers
14 views

PySpark using for loops

Questions Is it possible to use for loops (native) in pyspark 2.1? Yes --> How can I use these for loops? No --> Any chance to used on another way? In my case, I want to transform the following ...
0
votes
1answer
17 views

Regression in Python with many NaN values spread across all columns

I want to do a regression to predict "value" based on the other columns from below example table. The data was collected by single indicator and not across all data points, resulting in many NaN/blank ...
0
votes
1answer
17 views

Installed module pysubgroup not found in Jupyter Notebook

I'm trying to use the pysubgroup python package referenced here. I think I properly installed it as shown below, with no errors when I installed it: ...
0
votes
1answer
8 views

Determining the correlations between aggregated data and non aggregated data

I have a dataset which is essentially a list of lists produced by a sql query output. Here is what it looks like ...
0
votes
1answer
12 views

Classify members of a dataset, given a known group of members

Problem I have a large CSV file with lots of lines. It has 4 columns and each column contains a label and three integers. I know the first 10,000 lines belong to the same group. Now I need to ...
0
votes
0answers
9 views

Fitting a Hidden Markov model with Pyro

Hi everybody, I am trying to fit an HMM (hidden markov model) with the pyro, a probabilistic programming library. I generated a dataset to test my model with those rules hidden states : $z_{t+1} = ...
0
votes
1answer
12 views

Voting classifier using grid search for Time Series

I have three models Arima Auto ARIMA Double Exponential Smoothing I want to apply ensemble method - voting method and allow the classifier to learn weights for these three models. I have checked ...
0
votes
0answers
24 views

Which machine learning/deep learning model can I apply to a mix on textual, categorical and numerical data for a binary classification

I have a project based on tweets wherein I am trying to build a binary classifier, I am aware that I can use a contextual LSTM model which takes the metadata of a tweet as an auxiliary input within ...
0
votes
1answer
9 views

Comparing dates in 01/11/18 format with Python

Im new to this so please bear with me. I am trying to target items(rows) within my csv file with a date after 01/01/18 in that format. The file has 400k + rows with dates ranging from 2010 to 2018. ...
1
vote
1answer
9 views

Automatic legal document reviewing advice

I am lawyer from SK. I also like programming and Maths. Which technology should i use for this task? Automatic legal document reviewing. I have a big data set of specific contracts in slovak. Now I ...
0
votes
0answers
9 views

Survival analysis in xgboost

Does anyone know the expected format for survival analysis data in xgboost? The documentation states that you can select survival:cox as a learning objective but I ...
0
votes
0answers
6 views

Why does my custom categorical cross-entropy explode with Keras while not with TensorFlow?

I'm trying to train various Keras models on Pascal VOC 2012 dataset. The particularity of this dataset is that there is a particular class used to label "ambiguous" regions. These pixels are meant to ...
1
vote
1answer
13 views

Anomaly detection in nominal big data

I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
0
votes
1answer
16 views

Document similarity matching between Doc2Vec documents

I am creating a Doc2Vec model out of hundreds of PDF documents. I have 17 documents that are part of this Doc2Vec that I want to use to check similarity with other documents in the Doc2Vec model. ...
0
votes
1answer
26 views

Random Forest, Duplicating Data increases Accuracy. Why?

I duplicated my training data for the random forest classifier (Sklearn) and the accuracy of the prediction increased by about 3%. Why?
0
votes
0answers
6 views

What is the best tensor data format from saving in python and loading in c++

I hope to use Caffe c++ backend to conduct my new model training on a large embedding corpus data. As I am using python numpy to do the basic preprocessing and ...
0
votes
1answer
7 views

How to get the formulas used by seasonal_decompose for Trend and Seasonality

I'm trying to use decomposition to forecast into the future. From my reading I understand that I can do this by adding a trend formula to a seasonality formula. I know that I can decompose a time ...
0
votes
0answers
10 views

How to segregate clients on transaction and communication data

I have clients historical transaction data like date,stock data,quantity,revenue and communication data like sms and call logs.How to segregate clients using machine learning. i want to find similar ...
0
votes
0answers
17 views

Should I train entirely dataset or new part after I reload a ML model?

I don't know which one is the better way to do. I use scikit to train ML and save it. I train Model M with train data ...
0
votes
2answers
41 views

How to caculate time difference in between rows using loop in panda python

I have csv file with time and date. Here I want to calculate time difference row by row in time column. I wrote a code and when I run it it's not giving time difference row by row. Can anyone help me ...
0
votes
0answers
11 views

Protocol Buffers vs Hierarchical Data Format API performance

I have several models in .h5 file format. I need to expose them through REST API. Apparently, there are two options: Serve directly from ...
0
votes
1answer
24 views

Using categorial_crossentropy to train a model in keras

I'm a novice in machine learning. I was following this Keras blog to train image classifier using Keras. Though this blog only demonstrates how to train only two classes using binary_crossentropy, I ...
0
votes
1answer
11 views

How to train a model to predict a time window than an event will occur on a website

I have a list of historical timestamps of when a specific event occurred on a website. Currently the timestamp represents a 30 minute window that the event happened within. I am looking to train a ...
0
votes
1answer
20 views

Joining two dataframes on the basis of specific conditions [on hold]

I've added the images of both the dataframes here. Now I need to combine the two dataframes on the basis of two conditions: Condition 1: The element in the 'arrivalTS' column in the first dataframe(...
0
votes
1answer
7 views

Predict the probability that a user is active on a website

I have been given two different arrays, which describe if a user is active on a site or not based on some features. The first array is for training. Each row of the array is a different user and the ...
0
votes
0answers
12 views

Adaboost error in tuning with base estimator

I am implementing Adaboost for Stacking purpose and i define some methods to train and test model using kfold cross validation but the issue apprears when i used a base estimator for Adaboost i got ...
0
votes
1answer
10 views

Mixed effect random forest model for Python Windows

Does anybody know if there is a Mixed effect random forest model for Python Windows? The merf package https://anaconda.org/search?q=merf+ seems to only be available on a linux environment? thanks!
1
vote
1answer
41 views

'int' object is not iterable using panda python

Here I want to calculate time interval in between row by row in time column import from csv file. my start time 6.00 a.m and my end time is next day 6.00 a.m. In between this time periods how to find ...
0
votes
2answers
14 views

Keras y-labels range between 0 and 1 instead of binary?

I have X values and corresponding y-labels, until now I used to round my labels <0.5 to 0 and >0.5 to 1. Is it possible to use values between 0 and 1 for "y train"? Using Keras and Tensorflow. ...
2
votes
1answer
34 views

Keras - How to classify 1D time series

Say I have a training dataset composed by 128 1-Dimensional time series in form of numpy arrays. They all correspond to a certain action that I label action_1 and ...
0
votes
1answer
14 views

How to get different results running sklearn's MeanShift in a single program? (Python3)

I ran into a quirk with sklearn's MeanShift that I don't know how to get around. MeanShift doesn't predictably give the same results on every run, so I wanted to run it multiple times within one ...
1
vote
1answer
46 views

learning curve Sklearn

I was trying Random Forest Algorithm on Boston dataset to predict the house prices medv with the help of sklearn's RandomForestRegressor. Just to evaluate how good ...
0
votes
1answer
14 views

The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all() using panda python

enter preformatted text hereHere I have data to import from a CSV file.I wrote an equation inside the class and to solve the equation data will import from the CSV ...
1
vote
0answers
16 views

patient grouping and counting metrics spark

I am working on a project to define if kids have a given disease. The idea is to identify if the kids are sick and group them by disease. I have a large number of patients (kids) and I calculated ...
0
votes
0answers
6 views

Convert generator to DirectoryIterator in Keras

I created a multi input deep learning model, and a lot of functions I could not use it because the testgenerator (in the code) is a generator not a DirectoryIterator. ...
1
vote
0answers
10 views

Transformer architecture not working on toy problem

My transformer is not working on a toy problem. Toy problem Input : Sequence of random integer, one-hot-encoded. Example : ...
0
votes
1answer
25 views

how to find holiday effect on revenue?

I have 2 datasets from 2013-2017 for each day. a) Revenue generated by Locations and date. b) Holiday name and date I would like to know how each holiday is impacting the revenue by location. I am ...
1
vote
0answers
26 views

Programming a Neural Network in Python

In order to get some understanding of machine learning (i'm super new to this) I'm programming a neural network and try to train a sinefunction with it. The set-up is as follows: Backpropagation ...
1
vote
2answers
86 views

How can I implement tangent distance for k-nearest neighbor in python/scikit-learn?

My ultimate aim is to have a function which I can feed into scikit-learn's NearestNeighbor class as a custom metric parameter. ...
0
votes
0answers
15 views

Back propagation algorithm producing incorrect gradient in python

I am trying to implement a back propagation algorithm in python but I am finding that when I check this gradient against an approximated gradient the calculated gradient is wrong. I also found that ...
1
vote
0answers
17 views

Issues with pandas chunk merge

I'm trying to solve a kaggle competition - https://www.kaggle.com/c/ga-customer-revenue-prediction Since the data is too much to fit in memory at once, I'm trying to clean, process and save data back ...
0
votes
0answers
12 views

Ensemble of LSTM (for text) and machine learning models based on bagging or voting techniques

Are there any good kernels or blogs on ensemble of LSTM (on textual data) and machine learning models like Naive Bayes, Random Forest, Gradient Boosting etc. (on meta data of text) Ex data: ...
0
votes
1answer
28 views

Obtaining a confidence interval for the prediction of a linear regression

The data I am working with is being used to predict the duration of a trip between two points. There are about 100 different trips in the data and ~90k observations. I am using the standard pattern: ...
1
vote
0answers
14 views

Implementation of back propagation in python

I am trying to implement a back propagation algorithm in python but I am finding that when I check this gradient against an approximated gradient the calculated gradient is wrong. I also found that ...
0
votes
1answer
18 views

Forcing a multi-label multi-class tree-based classifier to make more label predictions per document

I'm been experimenting with tree based classifiers for multi-label document classification. All the trees I've created, however, tend to predict only one or two labels per document. Whereas the ...
0
votes
0answers
8 views

Multiply a dataset's column through Python in Orange Web Services

I have a dataset loaded into Orange that has many columns. The first column is called 'Time' and has values expressed in Seconds. I would like to know how can I use Python (within the Python Script ...
0
votes
1answer
37 views

Nearest Neighbors on mixed data types in high dimensions

I would like to be able to use nearest neighbors to attempt to find the most similar samples to a subclass of samples (think treated vs untreated) in a dataset with continuous, categorical, and text ...
-1
votes
0answers
18 views

How to calculate accurate values of parameters if I know the formula's form and many of its samples with python?

If we know a formula's form, but don't its parameters, E.g. y = a*(x1**3)+b*(x2)+c*(x3) I don't know the a, b, c, but I'm sure the variables of x1, x2, x3 and y's relationship should follow this ...
0
votes
0answers
21 views

Modeling data for Machine Learning

I am a beginner in machine learning and i struggle with understanding of how can i structure my data or get useful data to solve my problem or if it's even possible or needed with machine learning. ...