Questions tagged [numpy]

NumPy is a scientific and numerical computing extension to the Python programming language.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Varying a Numpy Array by a percentage [closed]

I want to use (5,5) numpy arrays to represent English letters, the following array is for letter "A" ...
0
votes
0answers
9 views

Parallelization of a MIMO linear filter

I would like to implement a Multi Input Multi Output filtering operation, acting as fast as possible on batches of data. Here is my current implementation: ...
0
votes
0answers
13 views

np.loadtxt function showing error, could not convert string to float

This is my data file : I am trying to read this file with loadtxt() and the geometry points get loaded into one NumPy array, so I separate the data into two arrays:...
0
votes
0answers
14 views

numpy elementwise multiply vector and matrix that have the same number of rows [migrated]

I have a matrix and a vector that both have n rows. I would like to have the kth row of the matrix be elementwise multiplied by the kth element in my vector, without a for loop! I have already tried ...
0
votes
0answers
9 views

fastai - using 'untar_data' function in kaggle kernel

I have recently started with fastai lesson 1 and I am using kaggle to run the course notebooks. While going through the ‘lesson1-pets’ notebook we use ...
0
votes
0answers
13 views

Building a gradient descent linear regression model from scratch on Python

I am trying to build a liner regression model using batch gradient descent to minimize the cost function. However, I am clearly doing something wrong because when I try to plot my cost history after ...
0
votes
0answers
16 views

Convert a pandas df with text to embedding matrix efficiently

What is the most efficient way of converting a pandas data frame with approx 5 000 000 rows, where there exists short texts in one of the columns to an embedding matrix? I am using the HuggingFace ...
0
votes
0answers
7 views

How to find average lag time with variance & confidence of two variables

I have two variables as time series, one a consequent of the other, I would like to find the average time delay it takes the dependent variable to act on the independent variable. Additionally, I ...
1
vote
1answer
29 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
0answers
16 views

How much can I reduce sample size in a bootstrap used for density estimation?

I have a dataset with 90m rows by 10 numeric/boolean columns. I need to calculate bootstrap distributions for 40 statistics. These statistics are definitely not Gaussian (nor similar to the mean of ...
1
vote
1answer
31 views

Why does averaging over vector values cause errors?

As a way to improve my model, I want to average GloVe vectors over a sentence. However, I can't get np.mean to work. The following code works when not averaging over words. (copied from other code) <...
0
votes
0answers
15 views

Calculate the correlation coefficient without one-hot encoding the category variable

I just read about Cramers V statistic to calculate the correlation coefficient without one-hot encoding the category variable. I'm trying to use it for finding the correlation coefficient of my target ...
0
votes
1answer
24 views

Getting the leave-one-out error on least square regression to fit polynomials

I need to implement least square regression to fit polynomials of degree 1-27. I then need to get the leave-one-out error (kfold cross validation where k = n). After doing a lot of research it seems ...
0
votes
0answers
24 views

Using numpy array as a feature in RandomForestClassifier

I am working on a bit of a variant to the standard image based object detection problem. “Standard” implying passing a trained model an image, and getting back bounding boxes, scores,…etc for all ...
-1
votes
1answer
24 views

Error in numpy array assignment

I'm trying to upload 17 images into a 4d numpy array, each image size is (256,256,1), so basically I'm using the 0th dimension for collection of different images. Following is my code: ...
1
vote
2answers
28 views

Replace Values with corresponding numbers in the same dataframe

Please help me to replace NR values with corresponding values as in the dataframe. Thanks, Phani.
0
votes
1answer
33 views

Neural network does not work on XOR

So, I am trying to implement a neural network in Python by only using NumPy. I have tried to do this by following 3Blue1Brown's video's about the topic, however, when testing my implementation, the ...
2
votes
1answer
26 views
0
votes
0answers
6 views

Handling numpy data in highly frequently updated environment

Having the following environment: Where the input rows (150 million per day) are getting directed into 500 independent DataFrames (300k rows per DataFrame). In ...
1
vote
0answers
28 views

Stratified K fold same index present in both test and valid set

I am trying to do a stratified k fold cross validation for my dataset and want to keep an isolated 10% test set from the dataset and use the remaining for training and validation. Below is the code ...
1
vote
1answer
20 views

Random Parameters to fix in ML to perform controlled experiments

Many algorithms and methods in modern Machine Learning techniques contain randomness, and because of that, running the same ML script several times can result in different outputs, therefore accuracy ...
3
votes
1answer
75 views

tensorflow pseudo inverse doesn't work for complex matrices!

The Tensorflow documentation here says that: tf.linalg.pinv is ''analogous to numpy.linalg.pinv. It differs only in default value of rcond''. However, tf.linalg.pinv requires the matrix to ...
1
vote
0answers
157 views

How to apply a groupby rolling function to create multiple columns in the dataframe

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo and the link to the data is here and the example here. Some Sample of data ...
2
votes
1answer
49 views

Combine 100 2D arrays into one 3D array

I have 100 2D lists a0 to a99, it is easy to convert them into arrays (I verified already): ...
2
votes
2answers
308 views

Ways to simulate weather data over several periods (Python or R)?

I have a time series dataset that has several variables for a state/province for fixed periods of time. That is for state A, there are samples from April 2017 to July 2019. Of course, I thought adding ...
1
vote
1answer
35 views

Implementing “full convolution” to find gradient w.r.t the convolution layer inputs

I've been trying to implement "full convolution" w.r.t to convolution layer inputs. According to this article, it looks like this: So, I wrote this function: ...
5
votes
4answers
711 views

Convert Numpy array with 'n' and 'y' into integer array of 0 and 1

I have a NumPy array of strings: 'n', 'y', wanna convert it into integer array of 0, 1, how to convert it? ...
2
votes
1answer
37 views

Machine learning dataframe dimension concept vs NumPy dimension

From Machine Learning for Absolute Beginners: A Plain English Introduction: Contained in each column is a feature. A feature is also known as variable, a dimension or an attribute - but they all ...
-1
votes
1answer
157 views

NumPy — ImportError: PyInit__multiarray_umath [closed]

I'm facing the below error while importing NumPy library. I have tried all of the suggestions advised by Python in its error log, but no luck. Strangely, when I use Python's OS library and print the "...
0
votes
0answers
22 views

ValueError while Comparing with Multilevel Index

My dataframe - ncg_vol contains Multilevel index - 'Year' and 'Month' and single column 'std'. I used .apply() on std column to perform operation based on the number of days in that year. I used the ...
0
votes
0answers
13 views

Input shape error for multi label segmentation

I am trying to perform multi-class segmentation and when I try to feed the model with my input images and model it get this error: ...
1
vote
1answer
29 views

Pandas dataframe with multiple observations per model

I currently have a pandas dataframe with the following format ...
0
votes
1answer
87 views

Trouble in calculating the covariance matrix

I'm trying to calculate the covariance matrix for a dummy dataset using the following formula, but it's not matching with the actual result. Let's say the dummy dataset contains three features, #...
0
votes
0answers
13 views

Group a dataframe or numpy array with cumulative value effectively

Suppose we have an array x = [1, 2, 1, 3, 2, 4, 0, 1, 1, 3] I want to divide it into sub-groups sequentially and in each sub-group the summation is just larger ...
1
vote
0answers
24 views

How to do Back Propagation Updation for below code?

Below is the code by Siraj Raval for implementing Neural Networks from Scratch. I have some doubts regarding the Code: Why during updation he did W2 = W2 + L1.T.dot(L2_Delta). I Mean shouldn't it be ...
1
vote
1answer
24 views

How should I improve my Vectorized Gradient descent linear regression model?

I wrote a vectorized Gradient descent implementation of the linear regression model. The Dataset looks something like: It's Not Working properly as I am getting negative R Squared error I don't ...
0
votes
0answers
30 views

How the Pandas Dataframe and Numpy masking technique works

I'm trying to understand how the masking technique works. I've a dataframe with some values (say one_row_incidents) some of the are not unique. By grouping them .by the duplicate index, I can get the ...
2
votes
0answers
22 views

Error when drawing random numbers from a custom continuous distribution using scipy.rv_continuous

I am trying to generate a sample of random numbers from a custom distribution $$ p(x) = x^{n}e^{-xtn}. $$ After reading the tutorial on scipy's website, I wrote a subclass which I called kbayes: <...
0
votes
1answer
28 views

empty fields to be filled with previous values

In the above sample data, I have empty fields and now the task is to fill the fields with previous values. my columns are dates and the values are a number of items present for that particular article ...
5
votes
0answers
80 views

Are there any graph embedding algorithms like this already?

I wrote an algorithm for generating node embeddings based on the graph's topology. Most of the explanation is done in the readme file and the examples. The question is: Am I reinventing the wheel? ...
0
votes
0answers
17 views

What types of variables are these? And what types of tests could I run?

So, I recently made a sheet of data collected from the 2010 Census on Cincinnati neighborhoods. The data is from demographics about the neighborhoods of the city such as median household income and ...
1
vote
0answers
52 views

Reshape Time series data for Conv2d Block

I am modelling my time series data into a supervised learning problem for the input to a conv2d block in pytorch from this tutorial. ...
0
votes
0answers
35 views

Data Analysis on normally distributed salary with Outliers

I have created normally distributed data with 2 outliers ...
0
votes
0answers
39 views

sess.run in losses.py

now I want to use sess.run in losses.py so that complex codes can be written by python style for customizing error function as below; ...
1
vote
0answers
21 views

Handling a large dataset consisting of npy files

I have a high number of npy files (448 files) each consisting of around 12k frames (150x150 RGB images) which together make the input to my neural network (X). However, since it is impossible to load ...
0
votes
0answers
51 views

For loop and assignment procedure in losses.py for customizing loss function of ML model

I want to customize a loss function for neural networks. A computation of the loss function requires the inclusion of a for loop like below. However, it is super difficult to implement because ...
0
votes
0answers
19 views

Fastest way to generate mini-batches from structured data

To generate mini-batches for training, I use a for loop to extract slices of a 2-D numpy array. Here is some example code I wrote to demonstrate this. Basically I loop through a 2-D numpy array ...
1
vote
0answers
34 views

Method for 'Simple' Clustering?

In a recent adult education class I took the prof shared a spreadsheet that was written in VBA code and takes ages to execute, I'm sure ...
2
votes
1answer
111 views

Numpy arithmetic operation between two columns

Below is the numpy array. I need to perform two operations on this array. Add one column with value [column 1] - [column 3]. Add another column with value [column 1] - [previous value of column1]. I ...
1
vote
1answer
39 views

Linear regression compute theta

I'm trying to compute the theta for a regression linear exercice. ...

1
2 3 4 5