New answers tagged data
0
There are a variety of ways to convert a categorical column to a numeric one, with the right answer many times being use-case specific. Trial and error can help here to see what works best for your problem.
To give a specific recommendation, you may want to try Target Encoding as an option and see how it performs. It will probably be better than One Hot ...
0
You can use label encoders for your product and market. however one hot encoding will be the best one to suit for converting string to numbers to feed into an algorithm . I suggest looking at this link for further details :- https://towardsdatascience.com/choosing-the-right-encoding-method-label-vs-onehot-encoder-a4434493149b
0
There are several approaches to this as you need both the input (images) and if your problem is a classification one, you need to reliably store the labels. You might also have some additional information about the images that could be useful for your problem:
you can store the images in such a way that all information is contained in the permanent store (...
1
There are many methods to connect two different kinds of datasets
Python Pandas - Merging/Joining
left − A DataFrame object.
right − Another DataFrame object.
on − Columns (names) to join on. ...
left_on − Columns from the left DataFrame to use as keys. ...
right_on − Columns from the right DataFrame to use as keys. ...
left_index − If True, use the index (...
0
See if this can help - Publicly Available Datasets
Also you can use SMOTE technique if you have insufficient data.
0
In reference with your little found data either augment it or apply cross validation on top of it.
else
Look for your expected data in https://datasetsearch.research.google.com/
1
This is a big topic with a lot of possible solutions depending on your context, I'll just provide some of my personal experience from projects I've built.
I would say that it is common in industry to retrain models in an ongoing fashion as new data comes in. Many times this happens at the daily level using either a basic cron-like solution or an enterprise ...
0
not sure if I understand correctly but it might be done with a command like this
df['category'] = np.where(df['category']!='0', '1', '0')
1
As already pointed out by the first comment, data science is a very broad field and not sharply defined.
But I think since you like to get started with data science and you're looking for resources, I share some of my recommendations with you (in no particular order).
Very applied, less academical, with lots of code examples: Hands-on Machine Learning with ...
3
Some feedback/tips/tricks/opinions here:
Problem setup
Including requirement analysis. Gotta decide how the system/solution should work, how to know ho how well we are doing, and then how to get there.
Model evaluation.
It is very desirable to have a quantitative way to evaluate our model performance.
For that we want some labeled data. It is very quick to ...
2
One quick thought is to use 1-NN (k-Nearest-Neighbors with k=1).
This does not need any training (it is a lazy algorithm) and for each new sample you look up what is the nearest datapoint in your dataset.
In your case, when you try to classify a new sample you can set its price to 0.
If you use Euclidian distance as a distance metric for the 1-NN, then in ...
1
Rationale
Some of the terms are a little vague, particularly what you refer to as eligible students and returned students. I'll set some variables for clarity, but tell me if I defined them incorrectly.
I assume them to mean:
eligible students $ = A $ being the set of all students in the after-school program 2019-2020
returning students $ = A\cap S $ where ...
1
I believe boxplot or violin plot is a good idea and you could overlay datapoints with a bit of jitter to the former. See below an example in seaborn taken from a relevant question:
import seaborn as sns
import matplotlib.pyplot as plt
tips = sns.load_dataset("tips")
sns.violinplot(x="day", y="total_bill", data=tips, color=&...
1
I can think of these possible solutions:
The basic one, club the whole data and try different algorithms and evaluate the results.
If Age distribution among the different samples(data set) are not proportionally distributed, i.e. if your dataset have huge samples of 'Young' in comparison to 'Adult' one or vice-versa, then I definitely try to tune my model ...
0
Hope this link would help you. It is always better to have more data, so concatenating the data sources would at least slightly increase the performance of you model than it used to be.
0
Seems to me the best way to do this is to train a single model on all of your data and let it sort out the features that distinguish one data set from another.
If you don't know in advance which set you're getting data from, you can't know which model to choose so having more than one would limit you
1
Add all data together, and make sure you have features representing all possible insights. In your case one feature with age/maturity (young, adults...). Lets say you fit a decision tree (or Random Forest, gradient boosting...) the model will decide whether to do a split or not on this feature if it contains meaningful information.
If you combine you should ...
0
If my specified model is predicting equally well (say in terms of classification accuracy) on two unrelated datasets; can I assume/conclude that the two datasets follow the same distribution? The parameters of the predictive model remain exactly the same in both the cases.
No you cannot, because there is no general equivalence between the two datasets ...
2
I think it would be better to use a standard scaler that removes the mean and divides by the standard deviation. See here for more info and an implementation using sklearn.
Why?
At least you should be aware that dividing by the maximum could hide smaller effects. In the case you have an outlier that has a very high value, you would loose the small changes in ...
0
We use column vectors because it offers efficient memory access in column-major format.
For example: CUDA, which is commonly used to accelerate libraries such as tensorflow, operates most efficiently when you coalesce memory. Strided memory access in parallelized applications cause significant slow-downs; threads (which are themselves sequentially indexed) ...
0
Alright, I believe I found one:
https://github.com/dchen236/FairFace
1
You can use inner join:
import pandas as pd
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')
df = pd.merge(df1, df2, on="Column1", how="inner")
Top 50 recent answers are included
