Questions tagged [numpy]

NumPy is a scientific and numerical computing extension to the Python programming language.

0
votes
1answer
27 views

How to determine input shape in keras?

I am having difficulty finding where my error is while building deep learning models, but I typically have issues when setting the input layer input shape. This is my model: ...
1
vote
0answers
25 views
3
votes
2answers
42 views

Is there a difference between np.matrix(np.array([0,0])) and np.matrix([0,0])?

I was reading this code, for implemnting linear regression from scratch: ...
1
vote
0answers
21 views

Integration of NLP and Angular application

I'm doing a small POC in which I've trained my Machine Learning model (Naive Bayes) and is saved in ".pkl" (pickle) format. Now my next task is to develop a web application which asks the user to ...
0
votes
0answers
15 views

Fixed: Keras Neural Network on handwritten digits labeling: Model reads the surrounding space instead of digit

Our assignment consists of labeling handwritten numbers, ranging from 0 to 9. We build a Convolutional Neural Network that achieves a good accuracy on our labeled testset. We now want to use this ...
1
vote
1answer
31 views

Is it a good idea to use tensorflow instead of numpy for numerical approximations?

I intend on perfroming some numerical approximations for a problem in physics. The main gist of the program will be to perform a svd on large sparse matrices and also calculating the trace of a ...
1
vote
0answers
23 views

Email clustering using numpy and pandas

Im trying to cluster my raw email data using numpy and pandas only. Hence Im Trying to create an email dataframe from raw emails that I have. However the issue that I have is that I keep getting an ...
0
votes
0answers
31 views

Implementation of Bhattacharyya distance for filtering images that are far off from their cluster

I need assistance with the python implementation of Bhattacharyya-distance for the below use case: Here, the polygons P1, P2..Pn refer to the different images where each pixel is represented by 'n' ...
0
votes
1answer
25 views

What is Difference Between np.zeros() and np.empty() [duplicate]

Can Anyone Please Explain me the Difference Between them In Terms Of Any Operation or Computation and Values Stored in them
1
vote
0answers
41 views

How to convert a TensorFlow tensor to a NumPy array within tf.Dataset.map() while multithreading/multiprocessing?

I use TensorFlow 1.12 in eager execution. I have a very expensive function which I map onto this dataset using tf.Dataset.map(). Inside this function — which I ...
0
votes
0answers
7 views

How to go from a tensor of 2D masks to a tensor of one-hot vector encodings for image segmentation in TensorFlow?

I use TensorFlow 1.12 in eager execution in order to perform semantic segmentation (one label per pixel) on images. My ground truth dataset is structured in instances however (multiple instances, or "...
1
vote
2answers
199 views

Is numpy.corrcoef() enough to find correlation?

I am currently working through Kaggle's titanic competition and I'm trying to figure out the correlation between the Survived column and other columns. I am using <...
0
votes
1answer
19 views

Is my iPython Installation Valid?

I just want to know if my ipython is properly installed. I have both Python and Anaconda installed on my Windows. I entered the "pip3 install ipython" command on the Command Prompt and it downloaded ...
0
votes
0answers
18 views

Using Matplotlib [duplicate]

I am new to data science. I use Anaconda on windows 7. I plotted a sine curve by doing the following on iPython: ...
1
vote
1answer
28 views

Naming the Axes of my Graph on iPython Using Matplotlib.pyplot

I am new to data science. I use Anaconda on windows 7. I plotted a sine curve by doing the following on iPython: ...
0
votes
1answer
33 views

What is .numpy()?

I searched for numpy.numpy() and tried replacing .numpy() with .np() because numpy is ...
0
votes
0answers
10 views

I am building a gesture recognition system using video data from ConGD Dataset and am not able to create the 5D array,as input to the network

I need a 5D input for my network using ConvLSTM and 3D-CNN. Converting the videos to an array of videos would contain a 5D array (Num of examples, Num of frames, FrameWidth, FrameHeight, NoOfChannels),...
7
votes
4answers
152 views

How to replace NaN values for image data?

My data set has a total of 200 columns, where each column corresponds to the same pixel in all of my images. In total, I have 48,500 rows. The labels for the data range from 0-9. The data looks ...
0
votes
0answers
15 views

Keras input shape error on predicting vectors

I have trained a Keras model (autoencoder) on embeddings that are 52,388-dimensional. I am trying to use that model to make predictions on a held-out data set that I am loading into Spark. The code to ...
2
votes
1answer
27 views

Model prediction on meshgrid in python

Suppose I have data with two independent variable $X_1$, $X_2$ and one dependent variable say $y$, as follows: $X_1$: $x_{1,1}$, $x_{1,2}$ , $x_{1,3}$ $X_2$: $x_{2,1}$, $x_{2,2}$, $x_{2,3}$ $y$: $...
1
vote
0answers
58 views

Cosine similarity with arrays contaning NaN

I am trying to calculate a cosine similarity using Python in order to find similar users basing on ratings they have given to movies. As it can be expected there are a lot of NaN values. I am using ...
0
votes
0answers
21 views

Issue in computing matrix inverse

I need to compute inverse of a matrix that has very small values of the range of 10^-9. When I use numpy.linalg.inv it gives all entries of the inverse matrix as 0. I checked in the original matrix ...
0
votes
0answers
78 views

Saving multiple numpy arrays to multiple image files

I have some CSV data that I have extracted in chunks of 375. Since I have 20 columns of data (7500 elements total), I have then reshaped these chunks into an image in the form of (3,50,50) to ...
0
votes
1answer
21 views

Decovolution function

I have an image (for example (7x7x3) and a filter (3x3x3)). I convolved the image with the filter and it became a (3x3) output. If I want to do the inverse operation and want it to become the image ...
3
votes
1answer
138 views

SVM SMOTE fit_resample() function runs forever with no result

Problem fit_resample(X,y) is taking too long to complete execution for 2million rows. Dataset specifications I have a labeled dataset about network features, ...
1
vote
1answer
36 views

What kind of algorithm should I choose for this music classification system?

I have in mind a program for analyzing short fragments of music, categorizing them as "good" or "bad". This would be part of a larger program that searches for larger good phrases and whole pieces. ...
1
vote
1answer
164 views

How to “reshape” into square matrix for numpy.linalg.solve()?

I'm trying to find the intersection of lines $y=a_1x+b_1$ and $y=a_2x+b_2$ using numpy.linalg.solve(). What I can't get my head around is how to correctly make $A$ ...
0
votes
1answer
32 views

implementing conv2d in fourier domain using np.einsum --> ValueError: einstein sum subscripts string contains too many subscripts for operand 0

According to the convolution theorem, convolution operation changes to pointwise multiplication in fourier domain - here I have 'fft_x' of shape (batchsize, height, width, in_channels) which is the ...
4
votes
2answers
85 views

Aggregate NumPy array with condition as mask

I have a matrix $b$ with elements: $$b = \begin{pmatrix} 0.01 & 0.02 & \cdots & 1 \\ 0.01 & 0.02 & \cdots & 1 \\ \vdots& \vdots & \ddots & \vdots \\ 0.01 &...
0
votes
1answer
49 views

Machine learning with sklearn vs. scipy stats

I've created 50 random x and y points (with slope of y = 2x-1). First, I used Linear Regression from sklearn to fit the model onto my dataset where I got a slope of ...
1
vote
0answers
426 views

How to fix setting an array element with a sequence error?

I am trying to make a keras functional model that solves Sentiment Analysis through the amazon database. I am trying to split the input data into 4 different tensors that can be individual feed in to ...
0
votes
1answer
34 views

Having difficult interpreting the eigenvectors for a simple 3x2 matrix

I calculated the eigenvectors and eigenvalues from a covariance matrix given a data matrix of 3 columns and 2 rows. I am trying to interpret results but I can't understand on how to interpret them. ...
1
vote
1answer
306 views

How feed a numpy array in batches in Keras

I have the data in the following format: 1: DATA NUMPY ARRAY (trainX) A numpy array of a set of numpy array of 3d np arrays. To be more articulate the format is: [[3d data], [3d data], [3d data], [...
0
votes
1answer
46 views

How to create new feature based on clustering result

I'm still a beginner in machine learning and I want to know how to code this situation based on python and machine learning (clustering). I have data like: ...
0
votes
1answer
256 views

Keras exception: Error when checking input: expected dense_input to have shape (2,) but got array with shape (1,)

I have an understanding of this error, it means that the input that I'm passing to the model is of a different dimension that what was expected. The error also states that the input that I'm passing ...
0
votes
1answer
48 views

How to get a dataframe values in one single column for the following dataset?

m38 m78 alpha 4.4717 4.8745 4.4569 4.6491 4.5101 4.7262 4.4407 4.8234 4.1184 4.3862 3.8448 4.2816 3.7246 4.6183 3.2857 4.6744 For the above sample data (actually 8000 rows) i ...
1
vote
1answer
107 views

SVR is giving same prediction for all features

I'm creating a basic application to predict the 'Closing' value of a stock for day n+1, given features of stock n using Python and Scikit-learn A sample row in my dataframe looks like this (2000 rows)...
2
votes
1answer
107 views

Fast way of computing covariance matrix of nonstationary kernel in Python

Suppose I have symmetric positive definite covariance function $k:\mathbb{R}\times\mathbb{R}\rightarrow \mathbb{R}$ that is non-stationary (i.e. $k(x,y) \neq g(|x-y|)$ for any function $g$). Is there ...
2
votes
2answers
81 views

How can I find similarities in two graphs?

Let's say that I have two 1 dimensional arrays, and when I plot the two arrays they look like this: If you look at the top and bottom graphs, then you can see that the highlighted parts are very ...
1
vote
0answers
15 views

How to do non arithmetic operation in python 3 [closed]

lets say a=3 b=4 and c is an unknown constant. a=3 b=4 F=0 F=a*b*c print(F) Its an error. I want 12*c or 12c
2
votes
1answer
127 views

Inputting (a lot of )data into a dataframe one row at a time

I'm using python. Some 2D numpy arrays are stored in individual rows of a Series. They are 30x30 images. It looks something like this: ...
2
votes
2answers
68 views

MAE,MSE and MAPE aren't comparable?

I'm a newbie in data science. I'm working on a regression problem. I'm getting 2.5 MAPE. 400 MAE 437000 MSE. As my MAPE is quite low but why I'm getting high MSE and MAE? This is the link to my data ...
1
vote
0answers
48 views

Two-sample K–S test Kolmogorov–Smirnov test in Python

For my evaluation, I have three different time-series data of the following format with different characterstics where the first column is timestamp and the second column is the value. ...
0
votes
1answer
21 views

Sequence to sequence RNN model, maximum number of training size

So when running this example script from Keras repo (https://github.com/keras-team/keras/blob/master/examples/lstm_seq2seq.py), I found that we can easily run into out of memory for the input or ...
1
vote
2answers
797 views

Numpy array from pandas dataframe

I am new in using python for data science. What is the difference between selecting a a column with: df['name'].values and ...
2
votes
1answer
62 views

Better code completion in python notebooks

How come python code suggestion is awful in notebooks and spyder? Smetimes it shows classes methods and variables, sometimes not
0
votes
1answer
26 views

Faster 3D Matrix Operation - Python

I am working with 3D matrix in Python, for example, given matrix like this with size of 2x3x4: ...
0
votes
0answers
39 views

Numpy Python deep learning framework

Does there exist a deep learning framework that is implemented only in Python and Numpy and supports 2D convolution, concatenation, ReLU activation function, max pooling, and global average pooling? ...
1
vote
0answers
531 views

IQR Score outliers detection in Python [closed]

I have a large dataset "train" incl. a column ClientFreeSource from which I want to exclude an outliers. For that purpose I decide to use IQR score. I've tried: ...
0
votes
0answers
135 views

How to create a complex Gaussian random noise with a specific covariance matrix

I am trying to generate a complex Gaussian white noise, with zero mean and the covariance matrix of them is going to be a specific matrix which is assumed to be given. Assume i to be a point on the ...