Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

How bit precision while training can impact DNN's accuracy - Libraries that would support inference with quantized types

I would like to check how bit precision impacts DNN's accuracy. Do you know any C/C++/Python libraries that wouldn't require huge rework for supporting inference with quantized types? For example, I ...
0
votes
0answers
8 views

(Py)Spark infrastructure for use in Java/C# website

Lets say I have experience creating ML models in Python, and have decided on training my models on Spark using Pyspark. This will form part of an ML infrastructure for a website with a Java or C# ...
0
votes
0answers
5 views

Help regarding generative modeling

I have data in form of n dimensional points. I have to train generative model where i pick a class through multinoulli distribution. Then each dimension of actual point is generated by some ...
0
votes
1answer
17 views

Free API for historical weather US data?

I am trying to retrieve a free R-Python API that provides historical weather data in US. In fact wunderground API is no longer available. Any suggestion?
1
vote
1answer
50 views

How to plot time series data from SQLite database

I am trying to plot time series data by accessing the SQLite database. I successfully connect the database by coding the below script,however, i couldn't figure out how to plot the time series data ...
4
votes
3answers
49 views

GridSearchCV vs RandomSearchCV and How it works?

GridSearchCV vs RandomSearchCV Can somebody explain in-detailed differences between GridSearchCV and RandomSearchCV? And how the algorithms work under the hood? As per my understanding from the ...
0
votes
0answers
11 views

How to save a trained ml model(e.g. decision tree) in database and load it using Flask to predict user submitted data?

I have tried the following way. It worked, but it is giving a different prediction in Flask vs Jupyter (which is correct) ...
1
vote
2answers
34 views
0
votes
2answers
27 views

Is it a best practice to exclude retweets from the data set?

I am going to build machine learning algorithm to identify fake tweets. The data set has huge retweets which I think might be an issue. Do you think given that the focus is the original tweet, it is ...
0
votes
2answers
39 views

CNN implementation low accuracy on MINST data

I'm trying to implement VGG11 (Model A of Table 1 from https://arxiv.org/pdf/1409.1556.pdf) on the MINST dataset but I'm getting ~10% train & test accuracy (as bad as random guessing). I had to ...
0
votes
0answers
56 views

Error loading a model (.h5 file) after training yolo-keras classification model

I am working on realtime object detection using my laptop's camera with Yolo and Keras. I have trained a model and the resulting output is a .h5 file containing (from my understanding) the model and ...
1
vote
1answer
37 views

How to calculate accuracy, precision and recall, and F1 score for a keras sequential model?

I want to calculate accuracy, precision and recall, and F1 score for multi-class classification problem. I am using these lines of code mentioned below. ...
1
vote
2answers
75 views

Replace data in Pandas dataframe based on condition by locating index and replacing by the column's mode

Hopefully, you don't mind me posting a question here instead of the regular stack exchange forum. I'm learning ML basics and practising pandas. assuming the data frame is called df, column name = ...
1
vote
2answers
26 views

How to optimize input parameters given target and scoring parameters

I'm new to machine learning/optimization, so I apologize in advance if this has been answered before. I don't know which search terms to use. I have a large dataset where I have a number of input ...
2
votes
2answers
35 views

Pandas datetime error when reading from excel file

I am trying to read an excel file that has two columns using pandas. This is how the data looks in excel file: ...
0
votes
0answers
22 views

3d density of population visualisation on map Python

I am looking for Python module, in which I could create 3d visualization of population density on map. To create 3d plot like this:
0
votes
1answer
14 views

fast processing: changing the column value in a geospatial environment

I have a table whose header looks like this: complaint_type, borough, street_name, incident_zip, latitude, longitude 1) I want to check if the "incident_zip" column of each row is in a specific list ...
1
vote
1answer
36 views

Is there any way to create column based on some previous column in PANDAS dataframe?

Given: I have Pandas Dataframe as shown below ...
0
votes
0answers
33 views

Data Analysis on normally distributed salary with Outliers

I have created normally distributed data with 2 outliers ...
0
votes
0answers
16 views

Can I use/modify an Autoencoder to handle missing data?

I am about to implement an Autoencoder to detect anomalies. Therefore, e.g., in my test set, there is a situation where the data stream broke for some days. This results in a lack of data and should ...
0
votes
2answers
28 views

How to get an intution for generating Keras CNN layers for a model

I am going through tutorial for handwritten text recognition. And to do hand written digit recognition the author has constructed a Keras model as follows: ...
0
votes
0answers
23 views

Preprocessing for Text Classification in Transformer Models (BERT variants)

This might be silly to ask, but I am wondering if one should carry out the conventional text preprocessing steps for training one of the transformer models? I remember for training a W2V or Glove, ...
-1
votes
0answers
27 views

Python Neural Network choose correct text

Hey I am new into the neural network scene. I would like to create a network which takes as input several file names. To train it I would say which files are usable and which are not. For example: ...
0
votes
0answers
40 views

UserWarning: `Sequential.model` is deprecated

C:\Users\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\engine\sequential.py:110: UserWarning: Sequential.model is deprecated. ...
0
votes
1answer
20 views

Target encoding before split on skewed data

Hi My data is distributed like follow: And I only have categorical variables on many many levels. As I need to make a regression task I thought about doing leave one out encoding on my categories. ...
0
votes
1answer
68 views

'list' object has no attribute 'values' when we are using append in python

Here I have a dataset with three inputs. Here I generated y value using append. After the append I got the output like this: y.append(rec.iloc[0]['y']) Then I ...
0
votes
0answers
38 views

How to define quadratic weighted kappa as eval_metric in catboost classifier

I am using catboost for a multiclass classification problem. I want to use quadratic weighted kappa as the evaluation metric. Catboost already has WKappa as an eval_metric but it is linearly weighted ...
1
vote
0answers
76 views

Fully-Configured Deep Learning Virtual Machines in Python (VirtualBox or VMware)

Fully-Configured environment setup for Deep Learning Virtual Machines in Python (VirtualBox or VMware) Often when we start working on any new technology, the most common challenge that we face is "...
0
votes
0answers
18 views

Using normal distribution data with high amplitude

I am using some data from a csv file to train a model which detects credit card fraud. The data set is from Kaggle and has 284,xxx samples each with 30 features. After some visualization in Python, ...
3
votes
2answers
214 views

Need machine learning algorithm to fill in time-series data

I am currently dealing with a time-series data set with cyclical gaps every 30 minutes (30 minutes of data, 30 minutes of no data). Is there a relatively simple way of using scikit-learn (or some ...
1
vote
2answers
27 views

Does the predict function in machine learning understand categorical data

I understand that before feature engineering one has to split the dataset into train and test data, so as to avoid bias in the analysis. I also understand that the machine learning model does not ...
0
votes
0answers
20 views

Clustering data using KMeans centroids of base period for pattern analysis

I have a data frame consisting of 12 months of Customer Transaction Level Data. The data is unsupervised. The data is divided into 6 sets of 2 months period each. Taking first period as the base, I am ...
2
votes
0answers
92 views

Python: Plotting equation with the Dirac Delta function

I am attempting to plot the expected diffraction pattern from a sharp razor which has a formula of: $$ I(p) = \frac 14 \delta^2(p) + \frac {1}{4\pi^2p^2}$$ where $p = \frac {\sin\theta}{\lambda}$ I ...
-1
votes
2answers
81 views

GridSearchCV for lightbgm classifier for multiclass problem

I am doing the following: ...
1
vote
0answers
27 views

Linear Regression on data with bimodal outcome

I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
2
votes
0answers
29 views

Did I do the right thing in my CNN Keras (class imbalance - augmentation)

To implement my Binary CNN in keras, I had a dataset of ~~35000 images but only 700 is from one class and all the others are from the other class, so what I did: I get the 700 unique images from class ...
0
votes
0answers
20 views

How do I get confidence intervals for an ElasticNet in sklearn?

I need to produce a row for the confidence interval for every field that I am calculating coefficients and scores off of. So here is my code so far- ...
2
votes
1answer
33 views

Dataset from sequence of messages

I have a sorted dataset by datestamp which looks like this: ...
0
votes
0answers
5 views

How to generate alignments for word-based translation models if number of words are different in both sentences

I am working on implementing IBM Model 1. I have a parallel corpus of some 20,00,000 sentences (English to Dutch). Also, the sentences of the two docs are already aligned. Aim is to translate a Dutch ...
0
votes
0answers
20 views

Should I remove the background of my training images?

The images in my dataset look are as below: The images have either a purple background or a white one. But the trained model (cnn) will be tested on images from the field ,that is, they will most ...
2
votes
0answers
57 views

Results are too good.. what is wrong? How to predict correctly?

I am about to evaluate a neural network and want to check whether the predictions make sense. The variables: ...
5
votes
2answers
277 views

How to install Polynote on Windows?

I've been searching around the Internet for a while but I have not been able to find detailed instructions on how to install Polynote (The polyglot notebook with first-class Scala support.) for ...
2
votes
1answer
43 views

What is difference between “cv2.filter2D” vs Keras “Conv2D” function

When I have to sharpen an image using opencv, I use: ...
0
votes
2answers
25 views

customer segmentation with unbalanced data

I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
0
votes
0answers
29 views

Time series forecasting produce same values with different training data

I'm developing a python program which predict daily timeseries values. Each daily timeseries contains 288 values (a record every 5 minutes). The main idea is to train a LSTM model with 7 days data ...
0
votes
0answers
13 views

Ho do I model spatial-temporal data with python?

Can someone get me started with the following: I have data of rare events including time and location. I want to predict future events based on past events occurring in a particular region and its ...
1
vote
1answer
15 views

help finding research discussion on HTS classification

My question is about the theory of this problem, and not necessarily syntax. I'm wondering if anyone here has experience with automating HTS (Harmonized Tax Schedule) classifications, specifically ...
0
votes
0answers
16 views

why evaluate_generator accuracy too much by using flow_from_directory with shuffle true?

i am using tensorflow and keras on colab i train with shuffled data but i met this strange accuracy with evaluate_generator with my training set that has 95% acc if i did not shuffle my data with ...
1
vote
1answer
38 views

How do I know how to construct the layers of my CNN

I've done a CNN project with Keras and OpenCV, and I've got roughly 65% accuracy. And now I have to present this work in my University, but I'm afraid if the teachers ask me for how do I knew how to ...