4
If You have only females in your dataset, adding gender feature to the model input will not improve it.
The technical explanation on why it won't help changes between models, but the intuition is simple - the model tries to find correlation between the features and the labels, and the correlation between any variable and a fixed-value variable is zero.
You ...
2
Psychology was mentioned because psychology has a long history of assigning numeric scores to subjective topics. One of the most important concepts is inter-rater reliability, how much do different people agree on an interpretation.
Other concepts that are useful are the degree of subjectivity and the degree of polarity (vs. assigning binary polarity labels)....
1
Keep subject_ID and after train_test_split pass to the model dataframes without the ID column, as in:
df.loc[ : , df.columns != 'subject_ID']
Unless you are explicitly shuffling datapoints during prediction, I believe that commonly returned predictions persist the initial order. Definitely worth checking with the particular model you are using.
EDIT:
See an ...
1
I completely agree with the answer on statsSE, I don't have much to add to it:
Essentially this is a business decision: you can voice your concerns if you think that the company is making a bad decision, but at the end of the day this is their choice to make.
There's one point in particular that I think is worth making clear to the company, it's what it ...
1
You are correct. They are not independent and they are positively correlated.
Let $A$ and $B$ (or $X$ and $Y$) be two events for stating general theorems and $M$ and $T$ be the events "customer purchases mozzarella" and "customer purchases tomatoes" in this specific example. We will use $\wedge$ to mean "and" so that $M \wedge T$...
1
Given you have two categorical variables and the associated contingency table,
one option is to calculate the joint and marginal probabilities:
$$ P = \frac{count}{total}$$
Probabilities
Tomatoes
No Tomatoes
Row
Mozzarella
0.4
0.1
0.5
No Mozzarella
0.2
0.3
0.5
Column
0.6
0.4
0.1
The probabilities then can be used to answer questions about the data - If ...
1
Market Basket Analysis is the way to go about it.
Market Basket Analysis is a technique which identifies the strength of association between pairs of products purchased together and identify patterns of co-occurrence. A co-occurrence is when two or more things take place together.
Market Basket Analysis creates If-Then scenario rules, for example, if item A ...
1
Market Basket Analysis is a technique which identifies the strength of association between pairs of products purchased together and identify patterns of co-occurrence. A co-occurrence is when two or more things take place together.
Market Basket Analysis creates If-Then scenario rules, for example, if item A is purchased then item B is likely to be ...
1
Provided you have the data already, and the data is labelled (i.e., split into the two classes $A$ and $B$), it makes sense to produce a number of visualisations to gauge what the model output would be.
If you start with traditional classification algorithms like logistic regression, then the model output is going to be the probability of belonging to a ...
1
You could simulate data and fit a model to it as if it were real data. there are packages and functions in R and Python to do this. You'd have to be very clear that the data is faked. You could then examine the model and produce graphs as if it were a real one.
This has the downside that it involves writing all the code and writing code to sim data, which ...
1
Look at your past experience. Even though you're a novice, you were hired as a data scientist, so you'll probably have some experience with data science projects. A simple binary classification problem with a few hundred datapoints can be solved in a productive afternoon, whereas a large project that requires significant upfront engineering for the ...
Only top voted, non community-wiki answers of a minimum length are eligible