Questions tagged [numpy]

NumPy is one of the fundamental packages used for scientific computing in the Python ecosystem. Specifically, NumPy is used for numerical computing and uses n-dimensional arrays.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

MNIST trained network tested with my own samples

I trained a Dense Neural Network with MNIST dataset in order to classify 28x28 images of numbers. Now I was trying to make it work with my own samples (I draw the image of a "7" in paint and ...
1
vote
1answer
17 views

When should we use jaccard score?

I am a newbie in Machine Learning, I trained a binary classifier for bank loan prediction through Logistic Regression. I measured the accuracy of it with two methods: accuracy score and jaccard index. ...
1
vote
1answer
36 views

Numpy array vs Pandas DataFrame when training [closed]

https://towardsdatascience.com/speed-testing-pandas-vs-numpy-ffbf80070ee7 (You can open the link in incognito if its locked). Numpy arrays are faster than DataFrame on normal mathematical operations. ...
1
vote
0answers
16 views

Problem with FFT in Python for smaller df

...
1
vote
2answers
76 views

ML : Found input variable with inconsistent numbers [closed]

I am trying an Retail ML project, but am stuck on the Error "Found input variables with inconsistent numbers of samples: [982644, 911]". I tired many thing & I know why this error occurs,...
1
vote
1answer
48 views

Generate max likelyhood masks in numpy [closed]

We have several "in_arrays" like in_1=np.array([0.4,0.7,0.8,0.3]) in_2=np.array([0.9,0.8,0.6,0.4]) I need to create two outputs like ...
0
votes
1answer
15 views

Where are the predicted results located in this model?

I am new at this and am pretty sure this is a stupid question, but here it goes: Where can I see the results of a model's prediction? I did this course about deep learning, followed the tutorial, ran ...
0
votes
0answers
6 views

Invalid value encountered in double_scalars and UndefinedMetricWarning: Precision is ill-defined

This question I asked in stack overflow but didn't receive any comments yet Question Link
0
votes
0answers
9 views

Calculating all near-tangential lines for inflection points of a time series

I have been thinking about this problem for a while and I'm curious if anyone knows of a good paper on this, has any ideas for algorithms or improvements to the framework. The task is to store the ...
1
vote
0answers
28 views

Numpy-like reduction of particular dimensions in Pandas DataFrame?

With a numpy multidimensional arrays, it is easy to reduce particular axes with functions such as np.mean or np.sum. For example,...
0
votes
0answers
62 views

ValueError: `validation_split` is only supported for Tensors or NumPy arrays, found following input: RaggedTensor

I have following inputs to be trained on a CNN: x = np.array(Images) y = [ [[0]], [[76., 5., 9., 1., 0., 0.], [54., 4., 10., 51.]] ] Since the ...
0
votes
0answers
8 views

pandas reduce time for na checks (is_valid_na_for_dtype / is_na)

I have a small project that I iterate over a dataframe and do stuff with this data.. I know that if I can I should avoid iterate the rows but I'm afraid that I must so I use itertuples. Now I want to ...
0
votes
0answers
64 views

Logistic Regression Model returning same output for all inputs

I made a simple linear regression model with a simple .csv dataset that had 2 categories. For reference, the dataset looks like a larger version of this Basically, it is a hobby classification ...
1
vote
0answers
10 views

How to compute similarity matrix for strings efficiently? [duplicate]

Here I'm trying to compute similarity between 1000 cross 10000 strings (using Levenshtein distance), I'm using a dataframe approach where you just need to compare n(n-1)/2 comparisons instead of n*n. ...
0
votes
1answer
17 views

How to convert post-padded sequence to pre-padded sequence?

I have numpy array as follows: train_x = [[1,2,3,0,0], [2,5,0,0,0], [2,3,0,0,0], [0,0,0,0,0], [0,0,0,0,0,0]] Now, I would like to transform it to as shown below: <...
3
votes
2answers
917 views

Cannot reshape array of size 12288 into shape (64,64)

I have an image I loaded with the image.load_img() function but when I try to reshape it I get this error: ...
3
votes
2answers
753 views

Which is meant by +/-9.2e18 years in timespan?

I was able to convert the 9.2e18 AD to a date, but I am confused about the exact date. Which date is 9.2e18 AD and and 9.2e18 BC? Time span (absolute) - [9.2e18 BC, 9.2e18 AD] i.e +/- 9.2e18 years ...
0
votes
1answer
29 views

Sub-word tokenization preprocessing to use transformer

I'm stacked with executing the sub-word tokenization preprocessing to use transformer. According to the tutorial on the article, I have executed the sample code. However, one function was not defined ...
0
votes
1answer
16 views
0
votes
0answers
12 views

Model on non-iid data performing badly

I am on this lecture about non-iid data where we generated a timeseries data using the function below: ...
0
votes
0answers
21 views

How can I get dark, median-brightness and bright distribution histograms from an image?

I'm trying to replicate an augmentation technique used in this paper. This is how they explain the procedure: [...] the augmentation technique was used to adjust the histogram because the intensity ...
0
votes
1answer
14 views

How to compare error metrics for model with and without Seasonality?

I am aiming to guage the difference in my model performance from using data with and without Sesonality removal. My approach to Seasonality removal is taking the log of the column data and then ...
0
votes
0answers
25 views

How to solve incompatibility between DeepPavlov, Tensorflow and Numpy in a Python script?

I am trying to run a script in Python 3.7 that makes use of DeepPavlov and Tensorflow==2.4.1. It seems that there is an incompatibility, because: DeepPavlov can work only with Numpy==1.18 Tensorflow==...
0
votes
1answer
28 views

What is the fastest way to run predictions on a big dataset using pandas, numpy and keras?

I am a bit new to the field of data science and could really use some help. I used a natural language dictionary to train and test an ml model using keras and tensorflow. It detects the sentiments in ...
0
votes
1answer
27 views

How to group by one column and find second occurance of values greater than a threshold

The Problem I have a pandas dataframe that contains series of people, the week number that a visit occurred, and their systolic and diastolic blood pressures. ...
1
vote
0answers
8 views

Predict pixels in optdigits data set

I'm using this dataset : https://archive.ics.uci.edu/ml/datasets/optical+recognition+of+handwritten+digits a dataset that consists of 65 columns , the last column is the label for 10 classes i.e 0,1,2,...
2
votes
1answer
22 views

SGDClassifier - Why do I need to use argmax instead of argmin to find the lowest threshold satisfying given precision?

I am an experienced programmer, but new to Python and data science. I am following Aurelien Gerone's book and I don't understand one thing. I create SGDClassifier and calculate its ...
2
votes
0answers
25 views

What would be a good randomization environment for data science?

I would like to know if there are any best practices to optimize random environment. Currently I use this simple structure in my config : ...
2
votes
1answer
1k views

Big array and MemoryError: Unable to allocate memory (in Python)

I am trying to create a predictive model using linear regression with a dataset that has 157673 entries. The data (in a csv file) is in such format: ...
1
vote
0answers
418 views

Py error could not broadcast input array from shape (244,244,3) into shape (224,224,3) arrays have the same shape (224, 224, 3)

this similar to other questions where there is an error related to "broadcasting" a np array with a shape of (244, 244, 3) and a np array with a shape of (244, 244, 4) but in this case both ...
1
vote
1answer
61 views

Computing SVD through Eigendecomposition of correlation matrix

I am following the excellent series on SVD by Steve Brunton from the University of Washington, on YouTube, but I have trouble interpreting his 4th video on the subject. If I understand correctly, he ...
0
votes
1answer
39 views

Is it possible to implement a vectorized version of a Maxout activation function?

I want to implement an efficient and vectorized Maxout activation function using python numpy. Here is the paper in which "Maxout Network" was introduced (by Goodfellow et al). For example, ...
-1
votes
1answer
171 views
0
votes
1answer
40 views

What representation of data should I choose, pandas, numpy or tensors to train neural network model [closed]

I am having a pandas dataframe divided to X_train, X_test, y_train, y_test by train_test_split and I am using it to train my ...
0
votes
0answers
20 views

Manipulate output of 1 keras model and use that as input to second model

I am trying to merge two Keras Sequential model. The aim is to get output from 1 model, run the output through a function and then use that as an input to another model. I don't even know whether such ...
1
vote
1answer
251 views

Changing a list of tensors to a tensor of lists!

I have a list of tensors created by [some_function(x) for x in something] and this list becomes like this ...
0
votes
1answer
31 views

Using Linear Regression to Learn Polynomial Regression

Let's start by considering one-dimensional data, i.e., $d=1$. In OLS regression, we would learn the function $$ f(x)=w_{0}+w_{1} x, $$ where $x$ is the data point and $\mathbf{w}=\left(w_{0}, w_{1}\...
1
vote
1answer
455 views

How can I create .nii (nifti) file from 3D Numpy array

I have a prediction numpy array. How can I make a .nii or .nii.gz mask file from the array?
1
vote
0answers
2k views

ValueError: Length of values does not match length of index [closed]

Why when I concatenate DataFrames my code shows this error? I trained a model to detect and classify 6 classes, and then store the coordinates and probabilities of these classes in a DataFrame, but I ...
1
vote
1answer
30 views

How does time needed for training differ between different batch sizes?

I've constructed a CNN in Python using Numpy, which is trained with mini batch gradient descent for MNIST digit classification. When training with a batch size of 1, the time needed for 5 epochs is ...
1
vote
1answer
7k views

TypeError: Image data of dtype object cannot be converted to float [closed]

I am trying to convert files stored in a CSV file dataset to grayscale. I run the code which worked on my previous dataset but I now get this error: TypeError: Image data of dtype object cannot be ...
0
votes
1answer
263 views

Read csv file and save images from the output

I have created some code that reads my CSV file and converts the dataset to a grayscale image. I want to know if there is any possible way to read through each row in the dataset and save each of the ...
0
votes
1answer
851 views

numpy array center of mass

I'm wondering how does ndimage.measurements.center_of_mass calculate the center of mass values for a two dimensional numpy array. For example let's say we have an ...
0
votes
0answers
15 views

What is the formula used by np.dot?

Dot products are pretty simple for 1- or 2-dimensional arrays, but anything beyond that is incomprehensible to me. I tried looking into numpy‘s dot function but the ...
0
votes
2answers
29 views

How match output (pred value) to input value

I'm working with data(with 4 columns which are p(product), M(name of the store)), I want predict the demand of store for that I sued SVR on the data by theses formulation: ...
0
votes
0answers
27 views

What are the use-cases for heapsort option in numpy.sort()?

Could anybody share experience about use cases of the heapsort option in numpy.sort() (or pandas.DataFrame.sort_values()) in the data science field? I mean if the <...
0
votes
1answer
385 views

How to show two pictures in one cell in Jupyter Notebook? (matplotlib) (python) [closed]

As a beginner, can someone please help me with this? Is there any way to show both pictures in a single cell's output? My output could only display one picture. Thank you for your time and attention! ...
1
vote
1answer
1k views

Replace values in dataframe with another dataframes values based on condition [closed]

I have 2 dataframes with the same columns names and rows number but the elements' values are different. I need to change the value of each element in the first dataframe to 1 if its value in the the ...
0
votes
0answers
86 views

Keras RetinaNet implementation and width, height order

I'm trying to use the excellent open source implementation of RetinaNet with keras (https://github.com/fizyr/keras-retinanet) and I've noticed a little detail that I would like to clarify. In the ...

1
2 3 4 5
7