Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
1answer
9 views

How do I sum the last three rows based on condition?

I have a table like this: Date Revenue Outlier Month 01/2018 333 No M0 02/2018 125 No M1 03/2018 136 No M2 04/2018 123 No M0 05/2018 496 No M1 06/2018 134 No M2 07/2018 223 Yes M0 08/2018 145 ...
0
votes
0answers
10 views

plot time series in text by python

I have a txt file (here) and i have tried to plot it with pandas, xarray of python I tried with ...
0
votes
0answers
16 views

Interpretation of Loss and validation Loss in Keras

I am building a model to predict one label by taking one feature as an input. The two variables seems to be strongly correlated. I wanted to build a (sequential) Neural Network model with Keras in ...
1
vote
1answer
12 views

Cross Validation with Artificial Neural Network in Keras

I needed to use cross-validation in the Neural network. I used kf = KFold(n_splits=5, shuffle=True) and ...
0
votes
0answers
20 views

Dividing dataset into Training, validation and test set

I have a dataset of 535 classes (258 194 images) and I want to divide it into train, validation and test set. I am a new learner in image classification with CNN, should I do that manually or is there ...
0
votes
1answer
7 views

How to do encode this target vector containing strings

Consider a target vector like ...
0
votes
0answers
2 views

Use tf.image.ssim_multiscale for a batch of images in sequential model

I am trying to use SSIM as loss value for my Keras Sequential model. The output value is a set of 6 images - (6, 32, 28, 3). I want to implement a custom loss function for the model.fit() function but ...
0
votes
0answers
13 views

How to improve machine learning model using 2+ datasets

I am building a supervised machine learning model which (for example) predicts heart failure (yes/no). I have two datasets from 2 different labs A and B, which both have decent distribution, aka it's ...
-2
votes
0answers
16 views

Non linear regression using scikit learn [closed]

What are the different non linear regression techniques and how many of them are available on scikit learn?
0
votes
0answers
9 views

Tf/Keras LocallyConnected layers

I recently stumbled upon these layers, and it's an interesting idea, but I'm not sure what one would do with it. What is the application of a Convolutional layer that doesn't share its weights? Where ...
2
votes
1answer
106 views

Follow Up Question - Why use fit when already have fit_transform? [duplicate]

This is a follow up question to: What's the difference between fit and fit_transform in scikit-learn models? I want to know why should we use fit at all when we ...
1
vote
1answer
17 views

Changing a list of tensors to a tensor of lists!

I have a list of tensors created by [some_function(x) for x in something] and this list becomes like this ...
0
votes
2answers
19 views

How to add epoch in sklearn LinearSVC?

I have a model that i need to train multiple times using epochs, i tried adding this code clf_svm.fit(train_features, train_labels, epochs=10, batch_size=64) and it ...
0
votes
0answers
10 views

Neural Network for hierarchical data?

Say I have a dataset of observations each belonging to some group/cluster $i(i=1, ..., N)$, i.e daily sales of $N$ different shops, plus some independent features. That is, if each group has $M_i$ ...
0
votes
0answers
9 views

Scalable Infrence Server for Object Detection

I have created a Django service (nginx + Gunicorn) for object detection models. For my case i have 50+ models with resnet 50 based back bone. Server Machine Specification: 16 CPU 64 GB Ram I have ...
0
votes
0answers
11 views

Open source interactive dashboards for python

I want to create a dashboard in Python for some graphical reporting. But the website should be deployable. I want something similar to Dash. I want to display plotly plots with interactive widgets. ...
1
vote
1answer
18 views

Predicting high frequency sparse time series data in python

I have a dataset of a couple of EV charging stations (10 min frequency) over 1 year. This data consists of lots of 0's, since there is no continuous flow of cars coming to charge but rather ...
0
votes
1answer
20 views

Binary Cross Entropy | Manual scalars

I am wanting to make print statements "showing my working out" of Binary Cross Entropy loss function, that works with ...
-1
votes
1answer
14 views

Problem with mord library function 'Logistic AT'

I wanted to run a an ordinal logistic regression on my bag of words. I have used the code below with logistic regression but now I have modified it for an ordinal logistic regression. However, when I ...
1
vote
2answers
207 views

How to create a big data frame in Python

I have a sparse matrix, $X$, created by TfidfVectorizer and its size is $(500000, 200000)$. I want to convert $X$ to a data frame but I'm always getting a memory error. I tried ...
1
vote
1answer
23 views

Are we allowed to transform the continuous target variable by creating a log transformation in order to have a normal distribution?

The following code gives the target variable Item_Outlet_Sales before transformation and Item_Outlet_Sales_log which is transformed ...
1
vote
0answers
13 views

How would you optimize this Binary Text Classification model further? The data set is large (40000 texts)

I'm learning about tensorflow/keras since a couple months. What are some methods to reduce overfitting in the later epochs or increase val. accuracy in general? train data dim (40000,70) (reddit ...
1
vote
0answers
13 views

How to resolve IndexError while doing Monte Carlo for 1000 runs? [closed]

Below code runs without any problem, however when I run the same code using Monte Carlo Analysis for 1000 runs, it gives IndexError. Can someone explain why this happens. Thanks ...
0
votes
0answers
16 views

Mutual Information Score for higher dimensional features?

Quick Background I am building a simple offline Motor Imagery classifier for a Brain Computer Interface system in Python and sklearn for educational purposes. I am following this pre-print. Here's a ...
0
votes
0answers
15 views

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) [migrated]

I am trying to tune the hyperparameters of MLP sequential model but getting an error while performing this task. I have tried degrading/upgrading the scikit-learn version and using ...
0
votes
1answer
16 views

Still have the same error after reshaping

I set my column 1 to 13 as X and column 14 as my Y. My data set has 14 columns and 401 rows including the parameters ...
0
votes
1answer
19 views

Mututal Information in sklearn

I expected sklearn's mutual_info_classif to give a value of 1 for the mutual information of a series of values with itself but instead I'm seeing results ranging between about 1.0 and 1.5. What am I ...
0
votes
1answer
14 views

Unbalanced training set from balanced data

I am looking to get an unbalanced training set with a given ratio of classA:classB from a dataset without regarding if it is balanced or not. The point is to analyze the influence of data imbalance on ...
0
votes
0answers
6 views

python kfda - methods fit and score

I'm struggling with my project, based on kfda package. The Code is here. ...
-1
votes
0answers
36 views

pandas count and show percentage [closed]

I have a df like this I would like to count the occurrence and also present in percentage, like below. I can easily do this in excel pivot table function, how to do this in python? note: the result ...
-1
votes
1answer
17 views

Chose most similar sample (mutivariat) [closed]

I have a list of 50-100 samples with 4-8 Variables (see below). I am trying to find a suitable algorithm to select the "most similiar" sample to a new sample that comes in (given parameters ...
-2
votes
0answers
16 views

Big data analysis with Python [closed]

If I have a big dataset, say 1TB, and want to explore and analyze it in the same way I do in my Jupyter Notebook. Is there a way to work in a Jupyter Notebook with large data like this? How would you ...
1
vote
2answers
16 views

How to properly use oversampling without inflating results?

I am using with a tiny private dataset (over 192 samples) with 4 classes. A preprocessing step is trivial in order to do any classification. Among feature selection and extraction techniques, i ...
0
votes
1answer
10 views

Can I cluster an aggregated data-set (grouped by) and apply dimensionality reduction?

I have data of sales, however it is in the millions, about 500M rows. I aggregate this data by factors such as location, shoptype, country_of_shop, cardtype, and then the aggregated statistic is: ...
-2
votes
1answer
10 views

What is this PIP syntax: “azureml-dataset-runtime[pandas,fuse]” [closed]

I came recently across this environment file: ...
0
votes
1answer
15 views

Are there any tree-based models that use a genetic algorithm to generate the trees?

I have a large dataset (195 features x 20m samples) that I have trained using XGBoost. I would like to see if a genetic algorithm can beat XGBoost since the data has so much noise it is prone to ...
2
votes
0answers
22 views

Storing multiple time series in a python pandas dataframe efficiently

I got different time series. Each time series contains different attributes, which have data entries at different points of time (timestamps). Each time series is labeled and has a label attribute. I ...
0
votes
1answer
36 views

Image classification with CNN Python

I'm working on image classification using CNN, my dataset contains more than 50 classes (50 folders) which represent the types of car parts, and in each folder we have vehicle brands, each vehicle ...
0
votes
0answers
10 views

aggregating multi output regression outputs vs single output (=top down) approach: is it worth it?

basically I have this problem where I need to forecast the sales of some stores. such stores have multiple product lines of which I have the split data (say Y1,Y2,Y3 where Y1+Y2+Y3=Y). I have also ...
1
vote
1answer
34 views

How to plot multiple bar charts with different infill and outline?

I have the CSV's of data. I want to plot the distribution of every attribute or a single one to compare the distribution from other CSV's, Since all attributes are the same but the distribution is ...
0
votes
1answer
45 views

How to plot the bar charts of precision , recall and f-measure?

I have used 4 machine learning models on a task and now I am struggling to plot their bar charts just like shown below in the image . I am printing classification report to get precision , recall etc ....
0
votes
1answer
33 views

How to make a gaussian distribution in python considering mean. variance. skewness and kurtosis?

np.random.normal(mean,sigma,size) allows to create a gaussian distribution based only on mean and variance. I want to create a distribution based on ...
0
votes
0answers
24 views

2D Z-score/Mahalanobis distance that includes a penalty for uncertainty

I have some 2D points and I want to assess their performance against the target point. When I was doing this in 1D, I took the Z-score Z = (x- mu)/sigma, but that ...
1
vote
1answer
17 views

Incorrect visualisation using Plotly

...
0
votes
0answers
22 views

Save and Load Simple Transformer Model

I have trained Text classifier using simpleTranformer.ai I am struggling to save and load the model in docker container. Please let me know how can I save the trained model and then load it into ...
0
votes
0answers
24 views

Extracting the day from a date column [closed]

Can anyone try to help me extract only the days from the Order Date column please? ...
0
votes
2answers
17 views

Warning when plotting confusion matrix with all sample of one class

I have two arrays: the first one with all the correct labels (they are all set to zero since each sample belong to the same class) and another one with all the labels predicted by my neural network. ...
0
votes
0answers
9 views

Visualizing post-hoc results of many days

I'm looking for ideas how to plot many Tukey HSD results. My origianl dataframe has 3 treatments. The values of each patient were measured every day for one month. One-way ANOVA has shown that there ...
0
votes
1answer
28 views

Why my classification accuracy is high for both training and testing data?

I have a dataset with 10 features and 1 binary classification target. I tested this dataset with decision tree classifier. I did some basic check like missing values but the data looks clean. My ...

1
2 3 4 5
105