Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
3 votes
2 answers
12k views

How to get a count the number of observations for each year with a Pandas datetime column?

I'm working on a column that I converted from a object to a datetime datatype in Pandas. I'm trying to get a count of how many of observations are there for each year. This is the column: ...
harshcode's user avatar
1 vote
0 answers
371 views

What would be the equivalent of R's mboost in Python?

I am looking for the Python equivalent of R's mboost package ( mboost ). Would that be xgboost?
user27875's user avatar
  • 123
3 votes
2 answers
5k views

A single column has many values per row, separated by a comma. How to create an individual column for each of these?

As you can see below, I have a column called code with multiple values per row, separated by a comma. How can I create a column for each of these codes and make ...
Programmer's user avatar
1 vote
0 answers
104 views

Time array ( 10 min ) for start time to next time in time series import from csv file using python

Here I have time series import from csv file. Start time 7.00 a.m next time is 8.00 a.m next time is 9.00a.m like that it will continue till to midnight. Then again next day it will start with time 7....
awa's user avatar
  • 97
3 votes
1 answer
528 views

Creating similarity metric with Doc2Vec and additional features

I have a dataset which contains many features. Each record is company that has many features. For example... Company A: Keywords - data, big data, tableau, dashboards, etc. Industry - Information ...
Nate Raw's user avatar
  • 133
1 vote
1 answer
520 views

Target data values are not evenly distributed

Data nature: I have features with 10 numeric type, and other 10 categorical, with a lot of values, at the end, using one-hot encoding I got a matrix of 600 columns. My problem is with accuracy ...
bacloud14's user avatar
  • 453
2 votes
1 answer
186 views

Convert a csv file so that all attributes will become columns with corresponding element [closed]

so I have this dataset in a csv file. I would like to convert it into a matrix form. The data currently looks like this: ...
user59764's user avatar
5 votes
1 answer
514 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
NewBee's user avatar
  • 51
4 votes
1 answer
4k views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
Minila S's user avatar
1 vote
1 answer
348 views

Batch data before feed into CNN network

I am working on a project to classify CT scan images using the CNN model, the image size is huge and I want to feed it into the network using the idea of batches, tried doing that with this pieces of ...
Hunar's user avatar
  • 1,147
2 votes
0 answers
104 views

Reduce number of vectors in dataset to achieve the "same average dimensions result"? [closed]

Edit for re-opening the question, I'll try to answer questions made by @user2974951: I have a large user preference statistics for trichotomic data sets. You can visualize each data trio as a 3D ...
Sanxofon's user avatar
  • 121
0 votes
1 answer
111 views

Predict compatibility of 2 people as boolean classification problem

How can I predict the compatibility of 2 people as a boolean classification problem? I want to know if below is an appropriate approach to modelling compatibility, or if I should be using "market ...
tim_xyz's user avatar
  • 187
1 vote
1 answer
614 views

Comparing XGBR with CatBoost performance

I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques....
mirik's user avatar
  • 113
2 votes
1 answer
6k views

LSTM train on varying length sequences

So I have around 600 classes and 17k sentences for text classification.The feature vectors are trained using word2vec. Vector_size = 100 ; LSTM hidden units= 100 I am using class weight for ...
Roma Jain's user avatar
  • 121
2 votes
1 answer
85 views

Coefficient of determination is close to 1 but the value of RMSE is large. What does it mean?

I am working with the DecisionTreeRegressor and trying to understand how well the data fits the model. I calculated both RMSE ...
Suhail Gupta's user avatar
0 votes
2 answers
132 views

Face Recognition Using HOG+SVM

This is my approach ...
Akhil Alexander's user avatar
14 votes
3 answers
21k views

Increasing SpaCy max NLP limit

I'm getting this error: ...
D500's user avatar
  • 241
1 vote
1 answer
226 views

Can I retrain my keras model on small dataset with some empty folders

I am working on a vehicle classification problem. I trained my model with 605 labels(folders) and 300k photos. After testing my trained model, I checked my wrong answers' training and validation ...
ibrahimozgon's user avatar
2 votes
2 answers
75 views

Is it acceptable for a forecasting model to predict moving averaged version of the data?

I'm working on a project in which I'm developing a precipitation forecasting model. When I try to predict the original data, the model (LSTM neural network) is not able to predict the peaks. This ...
hyTuev's user avatar
  • 277
2 votes
1 answer
456 views

Mismatching target dimension in Keras LSTM

I have built this network for text classification: ...
Simone's user avatar
  • 705
11 votes
2 answers
4k views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
Suhail Gupta's user avatar
1 vote
1 answer
6k views

Adding a new custom column in a python data frame [closed]

I need to add a new column in a python data frame that has the dates of January in a column. Each date is repeated 24 times in the column without any intervention. The total number of entries will ...
VishwaV's user avatar
  • 379
2 votes
1 answer
202 views

Getting rid of maxpooling layer causes running cuda out memory error pytorch

Video card: gtx1070ti 8Gb, batchsize 64. I had such UNET with resnet152 as encoder wich worket pretty fine: ...
Rocketq's user avatar
  • 143
1 vote
0 answers
901 views

Find partial derivative of softmax w.r.t logits in python

I have trouble implementing back propogation for multi class classification My neural network has 2 layers Forward propagation X -> L1 -> L2 weights W are initialized as random ...
Ajmal Rasi's user avatar
2 votes
1 answer
19k views

TypeError: '<' not supported between instances of 'int' and 'str'

I have the following code ...
TigSh's user avatar
  • 243
3 votes
0 answers
146 views

Feature Importance Python

My dataset has around 1000 features and 30k rows. All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
TigSh's user avatar
  • 243
3 votes
0 answers
462 views

Genarate one hour time interval array using pandas in python (import from csv) to predict next value

I am trying to generate one hour one hour time interval to predict next value according to my data set imported from csv file. Here according to the time it will give outputs include in x column. This ...
awa's user avatar
  • 97
1 vote
0 answers
127 views

How to calculate and print unseen time series values for LSTM after train, valid and test data

My dataset is time series data. It contains data of 10 packages. trainX = (1410,3,15), trainY= (1410,1), testX=(110,3,15), testY=(110,1). Input is three feature vectors of last three months and i am ...
Hazel's user avatar
  • 135
4 votes
1 answer
307 views

Can someone please explain what this sample function is upto?

So there is a function in Dino_Name_Generator at Deeplearning.ai notebook ...
thanatoz's user avatar
  • 2,405
1 vote
0 answers
98 views

How to load label data presented in raster format into Keras/Tensorflow

I would like to use a CNN network to segment 2 objects (binary: "0: object not present 1: object present") into shapes, but I have an issue with data. The training data is 150 images and in &...
aminevsaziz's user avatar
1 vote
2 answers
3k views

LSTM future steps prediction with shifted y_train relatively to X_train

I'm trying to predict simple one feature time series data with shifted train data. The source looks like this: ...
MR.QUESTION's user avatar
1 vote
1 answer
2k views

Prediction with unseen values in categorical variables

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using: ...
sectechguy's user avatar
1 vote
6 answers
226 views

How to statistically prove that a column in a dataframe is not needed

I have a pandas dataframe consisting of dimensions and features of a different fabric materials. I have several rows per product material type causing the dataset to seem very huge. From basic logic ...
Sudhi's user avatar
  • 171
1 vote
0 answers
79 views

Training a convoltion neural network for localization

I am new into convolutional neural networks and I am trying to build a cnn for localization of object on tensorflow. Currently I am trying to replicate a cnn similar to that of Alexnet and then want ...
user3449290's user avatar
16 votes
6 answers
8k views

Are there any good out-of-the-box language models for python?

I'm prototyping an application and I need a language model to compute perplexity on some generated sentences. Is there any trained language model in python I can readily use? Something simple like <...
Fred's user avatar
  • 403
2 votes
1 answer
520 views

Downsample GPS track

I am working with GPS track files (list of X and Y coordinates). I have tracks with a high sampling rate and want to downsample the track for easier handling. The obvious way would be to create a new ...
user2445254's user avatar
1 vote
1 answer
274 views

Boundaries of Reinforcement Learning

I finally developed a Game Bot that learns how to play the videogame Snake with Deep Q-Learning. I tried with different neural networks and hyper-parameters, and I found a working set-up, for a ...
maurock's user avatar
  • 284
1 vote
1 answer
2k views

Voice recognition with fourier transformation with audio input in python

First of all is using the fourier transformation even a good method for recognizing different speakers? I'm not sure if it could recognize a voice if the things that are said are different. I know ...
Jair's user avatar
  • 13
1 vote
2 answers
912 views

Label Encoder encoding the same class as two integers

First I have defined classes of a label encoder with the keys of a dictionary. Then I have used that label encoder to encode some strings. But for the same string, its giving different integers. Why ...
ravishankar's user avatar
1 vote
0 answers
207 views

Is there a way to use Orange on a remote server or in the cloud? [closed]

My question there is a way to run the Orange, not on a local computer, but on a remote server, even in the cloud? My notebook has only 8GB memory and limited number of CPU core, which is insufficient ...
obalasz's user avatar
  • 11
2 votes
0 answers
68 views

Continuously train CoreML model after deploy on Server?

I am working on face recognition with CoreML.I make a CoreML model with python Turi Create and put it on Server. User download it and compile on device but there is one problem if we want to do any ...
Neeraj Kumar's user avatar
2 votes
1 answer
1k views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
gazza89's user avatar
  • 266
-2 votes
1 answer
6k views

How do I add a column to a Pandas dataframe based on other rows and columns in the dataframe? [closed]

I've tried a lot of different methods, but I can't seem to find the right way to do this. I want to create a new column based on the time and ...
raulb's user avatar
  • 1
1 vote
1 answer
337 views

More weightage to a categorical feature for an Autoencoder model

I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
Ashwini's user avatar
  • 235
7 votes
3 answers
2k views

0.1 accuracy on MNIST fashion dataset following official Tensorflow/Keras tutorial

My goal is to classify products pictures into categories such as dress, sandals, etc. I am using the MNIST fashion dataset, following this official tutorial word-per-word: https://www.tensorflow.org/...
Nicolas Raoul's user avatar
1 vote
1 answer
2k views

What is the significance of underflow during parameter update using stochastic gradient descent?

Background I am using scikit-learn's MLPRegressor to learn a model with the following arguments: ...
hazrmard's user avatar
  • 328
2 votes
0 answers
643 views

Output size is too small for SpatialAveragePooling in Unet

Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
Rocketq's user avatar
  • 143
9 votes
1 answer
5k views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
Suhail Gupta's user avatar
6 votes
0 answers
217 views

Julia programming language [closed]

I've heard a lot of buzz for Julia programming language and how it's the future of AI. Should I buy into the hype and start learning Julia? Or do you think its better to stick to a better established ...
Jerome's user avatar
  • 346
1 vote
1 answer
503 views

Value extraction from a python dataframe [ problem statement specific ]

This is an update on one of the older posts of a friend of mine (That post stopped getting responses as I got a partial solution to my problem) and I'm reworking that problem statement. Link to that ...
VishwaV's user avatar
  • 379

1
98 99
100
101 102
134