# Analyzing country data. Does duplicating observations by population for regression make sense?

Question: While analyzing country happiness data via OLS regression, should I duplicate observations based on country population?

Example: If duplicating per million, the U.S. would have 327 observations and Denmark would have 6. Currently every country has 1.

If it matters...

y: Happiness score

X: GDPPerCapita, InfantMortalityRatePer1000, UnemploymentPercent, GiniIndex, etc.

Function to duplicate observations based on population:

def create_population_weighted_df(df):
df_pop = pd.DataFrame(columns=df.columns)
for index, row in df.iterrows():
for i in range(int(row['PopulationInMillions'])):
df_pop = df_pop.append(row)
return df_pop


Thanks.

I see what you're doing, and understand the thinking behind it, but don't think it makes sense with this problem. By assigning 300 variables the same values, you're effectively saying that each of these million people have the average economic output (i.e., if you're using GDPpercapita) for all the people in your dataset. That's a pretty misleading understanding of the distribution of that variable, and will lead to poor analysis. Same goes for other aggregate factors.

If you're looking to do a regression where those are incorporated into your analyses, using the unscaled population data as one of your features will include that information as one of the coefficients you return. So you can get back the result on happiness of having a million more/less people. No need to duplicate the data.

IMO, the duplication of data will lead to change of distribution of overall data. It can be checked by various plot.

Now coming to regression part, it will depend what kind of techniques you are using for performing regression.

The standard practice is remove the duplicate data.

I guess in your case it doesn't make sense to duplicate your observations as you are observing mostly indexed data, which are an aggregate or standardized measurement (e.g. ratio or means).

Edit:

Maybe this helps.

Is this even sensible? Your data is country wide, duplicating data by any measure will only skew your results and produce fake "accuracy". If you do not have enough countries/records for modelling maybe think about enriching the data set with a time series (e.g. the same entries over multiple years for each country).

I don`t know exactly what you are modelling but let's say an example could be predicting happiness based on economic data (GDP, etc.), development indices, etc.

In this case each country is exactly one record and duplicating data would be absolutely wrong here. There aren't 327 countries with the statistics and happiness of the USA, there is only one. Multiplying by 327 will fake model accuracy and absolutely overfit to boot. Do not do it!