Danny
  • Member for 3 years, 10 months
  • Last seen this week
  • Dublin, Ireland
Why this sequential model is not starting?
Accepted answer
6 votes

from keras.models import Sequential from keras.layers import InputLayer model = Sequential() model.add(InputLayer(input_shape)) model.add(BatchNormalization(axis = 3)) This should work. The first ...

View answer
Discrimination vs Calibration - Machine Learning Models
Accepted answer
5 votes

Discrimination is the separation of the classes while calibration gives us scores based on risk of the population. For example, there are 100 people that we’d like to predict a disease for and we ...

View answer
Risk prediction vs classification model
Accepted answer
5 votes

I will try to answer your question as shortly as possible. Yes, if you define probability as a risk, then the probabilities are risk scores. But, there's a catch in these scenarios, you will have to ...

View answer
Mapping column values of one DataFrame to another DataFrame using a key with different header names
5 votes

df3 = pd.merge(df1,df2,left_on=['cat'+str(i)], right_on = ['cat_codes'], how = 'left') I would iterate this for cat1,cat2 and cat3. This does not replace the existing column values but appends new ...

View answer
Probabilistic gold standard vs Deterministic gold standard
Accepted answer
4 votes

No, deterministic probability is when you know for certain. If a person does not have a diagnosis, then he doesn't have the disease/condition. Doctors are not supposed to give a probability but we as ...

View answer
Classification problem with many images per instance
3 votes

I assume every instance is a grouped data of either hotels or hostels and both. I think this paper discusses your problem, provides solutions and comparisons between different frameworks. Instead of ...

View answer
How to convert sequence of words in to numbers which are input to RNN/LSTM?
Accepted answer
3 votes

from keras.preprocessing import Tokenizer samples = ["grss is green and sun is hot"] tokenizer = Tokenizer(num_words=1000) tokenizer.fit_on_texts(samples) sequences = tokenizer.texts_to_sequences(...

View answer
How to replace a part string value of a column using another column
3 votes

data['Product Name'] = data['Product Name'].str.replace('\d+','') This should get rid of the number if that's what you are looking for. I am not sure what you mean by 'chomped.'

View answer
Conceptual question on CNN and any multi layer neural network (Part 2)
Accepted answer
3 votes

There's an amazing book called "Deep Learning with Python" by Francois Chollet that you could refer to. To answer your question: You usually add high number of layers and check where the validation ...

View answer
Pandas throwing "Error tokenizing data. C error" while loading data sets from URL
Accepted answer
2 votes

The path that you are accessing from is a Github repository page which is a webpage, it does not return CSV. You have to click on 'raw' option in Github and then pass the URL which in your case is: ...

View answer
Unbalanced training data for different classes
2 votes

You can duplicate the images and add them. You can use data augmentation techniques for the labels which have less images. The below code is for Keras. datagen = ImageDataGenerator( rotation_range=...

View answer
Which recommender system: Content based or Collaborative filtering?
2 votes

You are right. Content-Based Recommender System is the best approach to tackle such problems using historic information about the users.

View answer
Why to exclude features used for label generation during modeling?
Accepted answer
1 votes

You created the labels using the data. If you are able to label them with the data, then why do you need a machine learning model? It simply becomes a rule based classifier. What you would like to do, ...

View answer
How to select one record from multiple record for a subject during analysis?
Accepted answer
1 votes

I will try and be as concise as possible. First, let's redefine the way you think about your data points. There can ever only be two types of visits in terms of time. Periodic and Non-Periodic. Let's ...

View answer
Doc2vec to calculate cosine similarity - absolutely inaccurate
1 votes

I think you are missing the model.infer_vector(new_sentence). You need to infer the new vectors based on your trained model. You can find more details in Assessing the Model section here. The ...

View answer
images from training set are different from images of test set
Accepted answer
1 votes

The way you use ImageDataGenerator is wrong. The .fit() method is trying to read in the directory path which is a string. To be able to run your code, you should remove the train_datagen.fit(...

View answer
How to arrange the sets to predict y on x in time series?
Accepted answer
1 votes

fig, ax = plt.subplots(1, 1, figsize=(24,7)) for train_index, test_index in tscv.split(X_train1_raw, y_train1_raw): X_train1, X_test1 = scaler.fit_transform(X_train1_raw[train_index].reshape(-1,1)...

View answer
Padding the sentences is consuming huge memory
1 votes

Assuming you have tried alternatives like storing it in csr matrices in scipy, you can move away from padding to avoid memory issues by declaring your batch_size=1 and may be create batches using a ...

View answer
Delete/Drop only the rows which has all values as NaN in pandas
1 votes

This should do it: df.dropna(axis = 0, how = 'all')

View answer
Identifying if the sentence if it comprise information about education
Accepted answer
1 votes

I think what you are looking is to differentiate between 'be' and 'BE' based on context. Word2Vec is a good place to start but to determine the difference between words based on contexts is 'Sense2Vec....

View answer
Seaborn barchart for frequency of data
Accepted answer
1 votes

This should do the trick. You have to melt your data frame to use x,y and hue in your seaborn barplot. yfreq['type'] = yfreq.index yfreq = yfreq.melt(id_vars = 'type') sns.barplot(x = 'variable', y = ...

View answer
How does the naive Bayes classifier handle missing data in testing?
1 votes

You tend to avoid these situations while preprocessing your data. You impute the missing data. In production terms, frameworks like H2O handle quite elegantly. If you mean that there's a dimension ...

View answer
Use prediction as feature for a decision tree
1 votes

Yes. Stacking is essentially feeding the predictions of the base learners to a meta learner. Sort of like a model of the models. Here's a good explanation of that. Bagging,Boosting and Stacking

View answer
Dataset with features as SNOWMED codes and outputs as names of medications prescribed.What are some ML algorithms that suit this case?
1 votes

In that case, Multi Class Classification will be the best approach. You can use Neural Network approach or K-Nearest Neighbours for this problem. Your DataFrame is going to be really sparse if you ...

View answer
Select more or less features if results are almost the same
1 votes

It is very contextual. There's a rule that feeding all the data will yield better (unbiased) results but not at all times. If you know anything about the data, then it is always advised to remove the ...

View answer
Why do we need a gain ratio
Accepted answer
1 votes

I had the same doubt when I was doing my Master's Degree. First of all, you don't include something as random as 'IDs'. This is where data preprocessing comes in. Let's take a dataset which has users ...

View answer
what are the next step after ML prediction and how to proceed?
0 votes

If one understands the business problem you are trying to solve, then this shouldn't be that hard. You can present the results of your model. Understand your data first, is it skewed, balanced? ...

View answer
What inference can we draw from the frequency distribution of thresholds?
Accepted answer
0 votes

I'd say the model 1 is performing really well. If you can use different colors while plotting for positive and negative classes, then you should be able to see the difference. When you are trying to ...

View answer
Varying the number of neighbors in nearest-neighbors regression
0 votes

The most common method that's used is "The Elbow Method." You decide your neighbours based on the error rate. You can use GridSearch to determine the ideal number of neighbours to be used. This gives ...

View answer
ValueError: Error when checking input: expected conv2d_1_input to have 4 dimensions, but got array with shape (142, 1)
Accepted answer
0 votes

def ReadImages(Path): LabelList = list() ImageCV = list() classes = ["nonPdr", "pdr"] # Get all subdirectories FolderList = [f for f in os.listdir(Path) if not f.startswith('.')] ...

View answer