Hot answers tagged

11

It is quite trivial (X=='y').astype(int) Should do the trick. It simply converts your array to True or False according to your requirements and then astype will impose the required datatype. By default int will give you 1 for True and 0 for False.


9

There may be column of type object in your dataFrame. Remove it or convert it into float,int etc. if possible From Documentation of Seaborn: data : rectangular dataset 2D dataset that can be coerced into an ndarray. If a Pandas DataFrame is provided, the index/column information will be used to label the columns and rows. so you can try sns.heatmap(...


8

Given you have images stretched out as columns in a table with ~48,500 rows, I am assuming you have the raw images that are 220x220 in dimension. You can use a function available via OpenCV called inpaint, which will restore missing pixel values (for example black pixels of degraded photos). Here is an image example. Top-left shows the image with missing ...


8

Seaborn uses inter-quartile range to detect the outliers. What you need to do is to reproduce the same function in the column you want to drop the outliers. It's quite easy to do in Pandas. If we assume that your dataframe is called df and the column you want to filter based AVG, then Q1 = df['AVG'].quantile(0.25) Q3 = df['AVG'].quantile(0.75) IQR = Q3 - ...


7

It works for me. >>> import numpy as np >>> dx = np.matrix([[ 1.6, 3.6, 0.4, 14.4, 25.6], ... [10. , 10. , 0.4, 14.4, 3.6], ... [ 0.4, 0. , 0. , 1.6, 10. ], ... [ 6.4, 0. , 3.6, 1.6, 0.4], ... [14.4, 0. , 25.6, 0.4, 6.4]]) >>> 10 * dx matrix([[ 16., 36., 4., 144., 256.], [ 100., 100., 4., 144., 36.], ...


7

I got your problem like this way: You want to show labels on x and y axis on seaborn heatmap. So for that sns.heatmap() function has two parameter they are xticklabels for x-axis and yticklabels for y-axis labels. follow below code snippet import seaborn as sns # for data visualization flight = sns.load_dataset('flights') # load flights datset from GitHub ...


7

I suggest you to visit this link also for this case would work np.reshape((-1,2))


7

You have already computed that, but you've not bound the output to a variable, also called name in python. Try the following snippet: result = np.linalg.norm(v1,ord=2,axis=1,keepdims=True) print(result) Based on the edit, I update the answer. As you may find answers to your question, a typical way to find what you need is something like the following ...


5

nazz's answer doesn't work in all cases and is not a standard way of doing the scaling you try to perform (there are an infinite number of possible ways to scale to [-1,1] ). I assume you want to scale each column separately: 1) you should divide by the absolute maximum: arr = arr - arr.mean(axis=0) arr = arr / np.abs(arr).max(axis=0) 2) But if the ...


5

I would import the datasets in pandas separately, mold them as you please, and then you can use the pd.concat function. This will assume that the instances are aligned by the automatically assigned index in pandas. If there is more data in one list than the other, the missing values will be NaN. df1 = pd.DataFrame(data=[1,2,3]) df2 = pd.DataFrame(data=['a','...


5

You could use the following code: X[X=='y'] = 1 X[X=='n'] = 0 This replaces the indexes of 'y' with 1 and of 'n' with 0. Generally the X=='y' returns a Boolean array which contains True where the 'y' and False everywhere else and so on.


5

You can compare the predictions with the expected results directly, using simple comparisons, in this case just ==. This returns boolean values - True or False, which you can sum up because True == 1 and False = 0. Here is an example for your case using some randomly generated dummy data: In [1]: import numpy as np ...


4

You don't need to load the whole dataset into memory at once. The only data you need in memory are the samples in a single training batch. Use the fit_generator method rather than fit to pass in an iterator that feeds samples to your model from disk rather than loading all of that data at once. Here's a tutorial that discusses this more.


4

Found the answer. Thank you @Aditya import seaborn as sns sns.lmplot('Time', 'Amount', dataset, hue='Class', fit_reg=False) fig = plt.gcf() fig.set_size_inches(15, 10) plt.show() where Time and Amount are the two features I needed to plot. Class is the column of the dataset that has the dependent binary class value. And this is the plot I got as required.


4

One approach is to plot the data as a scatter plot with a low alpha, so you can see the individual points as well as a rough measure of density. from sklearn.datasets import load_iris iris = load_iris() features = iris.data.T plt.scatter(features[0], features[1], alpha=0.2, s=100*features[3], c=iris.target, cmap='viridis') plt.xlabel(iris....


4

I don't understand why you would like to fill values with zeros ! This would basically mean, "this guy, who is 170 cm tall, weights 0 kg" and would fool your network. In my opinion, you have two options: discard missing values (the entire row): you end up with less but more consistent training data if you really need these rows, then fill missing values ...


4

Suppose you have an array arr. You can normalize it like this: arr = arr - arr.mean() arr = arr / arr.max() You first subtract the mean to center it around $0$, then divide by the max to scale it to $[-1, 1]$.


4

I still would apply numpy's covaranice function using numpy.apply_along_axis import numpy as np x = np.array([[0, 2], [1, 1], [2, 0]]).T np.apply_along_axis(func1d=np.cov, arr=x, axis=0)


4

This is expected and is not related to SMOTE sampling. The computational complexity of non-linear SVM is on the order of $O(n^2)$ to $O(n^3)$ where $n$ is the number of samples. This means that if it takes 0.8 seconds for 7.5K data points, it should take [3, 48] minutes for 115K, $$[(115/7.5)^{2} \times 0.8, (115/7.5)^{3} \times 0.8]s=[3,48]m,$$and from 16 ...


4

On a side note, I don't think correlation is the correct measure of relation for you to be using, since Survived is technically a binary categorical variable. "Correlation" measures used should depend on the type of variables being investigated: continuous variable v continuous variable: use "traditional" correlation - e.g. Spearman's rank correlation or ...


4

To check if a value has changed, you can use .diff and check if it's non-zero with .ne(0) (the NaN in the top will be considered different than zero), and then count the changes with .cumsum, like this: df['counter'] = df.diff().ne(0).cumsum() Afterward, you can create a second dataframe, where the indices are the groups of consecutive values, and the ...


3

I think I know what you are going for. Here is some example code. # Import pandas import pandas as pd # Load data df_a = pd.read_csv('gdp.csv') df_b = pd.read_csv('le.csv') # Use melt to pull the columns from each df into separate rows tied to the key column. df_a = df_a.melt(id_vars='country', var_name='year', value_name='gdp') df_b = df_b.melt(...


3

MemoryError is exactly what it means, you have run out of memory in your RAM for your code to execute. When this error occurs it is likely because you have loaded the entire data into memory. For large datasets you will want to use batch processing. Instead of loading your entire dataset into memory you should keep your data in your hard drive and access ...


3

If your purpose is to deal with some data analysis from your arrays and have a nice view of it, try this using pandas: import pandas as pd df = pd.DataFrame({'distA' : np.array([0.2, 0.3, 0.4]), 'distB' : np.array([0.3, 0.4, 0.5])}) df In the case you want to make computation with vectors, matrixes, tensors, etc .. NumPy is probably a ...


3

You should implement a generator and feed it to model.fit_generator(). Your generator may look like this: def batch_generator(X, Y, batch_size = BATCH_SIZE): indices = np.arange(len(X)) batch=[] while True: # it might be a good idea to shuffle your data before each epoch np.random.shuffle(indices) for i in ...


3

It looks fine to me :) the only problem is that your plot (resulting from In [18]) is being displayed on your computer in a separate window somewhere - maybe you have to find it. Once you close that window, your iPython prompt woill return to In [19]. You could alternatively press Ctrl-C in the iPython session, but this will end the session. If the problem ...


3

No, they're absolutely the same. In this case there is absolutely no difference apart from perhaps a trivial amount of processing time. This is all open source code so we can just read it: The relevant part of numpy for us here is the matrix constructor (yes, np.matrix is a python class below the hood). In a summary from the NumPy code we see: class ...


3

The problem is that train_test_split(X, y, ...) returns numpy arrays and not pandas dataframes. Numpy arrays have no attribute named columns If you want to see what features SelectFromModel kept, you need to substitute X_train (which is a numpy.array) with X which is a pandas.DataFrame. selected_feat= X.columns[(sel.get_support())] This will return a ...


3

Any scalar multiple of an eigenvector is also an eigenvector. LAPACK (which np.linalg.eig uses under the hood) chooses to return unit-length eigenvectors (good for SVD!), but this still leaves two choices, and there doesn't seem to be a convention for which one to return; it's up to the underlying algorithm (which in turn may depend on the input data). ...


3

This might Pandas convert a column of list to dummies. You can try using the MultiLabelBinarizer class from sklearn.


Only top voted, non community-wiki answers of a minimum length are eligible