New answers tagged

1 vote
Accepted

Making a netcdf data using xarray

...
  • 120
0 votes

How can i implement an confucion matrix?

Confusion matrices are supported by scikit-learn (see also lya Lees answer). In cell 1, you can import it via ...
  • 508
1 vote

What techniques are used to analyze data drift?

One way to start is fundamental exploratory data analysis. Compare univariate, bivariate, and multivariate distributions between training data and new data. Those comparisons can be done visually, ...
2 votes

What techniques are used to analyze data drift?

It depends about what type of data are we talking: tabular, image, text... This is part of my PhD, so I am completely biased, I will suggest Explanation Shift. (I would love some feedback). It works ...
0 votes

How can i implement an confucion matrix?

Without having read too much your code, a confusion matrix states how many elements from class $y_1, y_2, y_3..$ have been associated by the model to class $y_1, y_2, y_3..$ So, for n classes, the ...
  • 120
0 votes

CIFAR-100: What is the difference between vehicles 1 and vehicles 2?

You could create your own version of CIFAR 100 to do a CIFAR 10 Vs. CIFAR 100 experiment. Meaning that to a model trained on CIFAR 10, where it would learn images on Trucks, pickup Trucks are ...
0 votes

Why does PowerBI take longer to load data than PowerQuery takes to load preview?

Previews in Power Query usually filter to 1000 rows. They aren't pulling all the data. Adjust your timeout or shrink the number of rows and columns you pull back.
  • 101
0 votes

using a feature that is only available during training

Using the comments doesn't really make sense if they will not be available for prediction. If you add the feature you will end up having to impute the comment features in some way. How do you plan on ...
1 vote

How to Manipulate a Dataset

From this dataset, you should prepare new variables namely, X and y; where X = feature matrix...
0 votes
Accepted

Survey on image retrieval datasets

You can check in papers with code, in the "image retrieval" category: There, you will find multiple datasets and, for each one, the best-performing models according to certain metric, with ...
  • 18.9k
0 votes

CWRU Bearing fault

Fine-tuning a pre-trained model is indeed transfer learning.
  • 18.9k
0 votes

Accuracy on Validation and Test set, Overfit?

The existing answer is a good one - as it mentions you're likely underfitting. What type of model(s) are you using? If for example you're using Linear Regression you might opt for Polynomial instead....
0 votes

Open source NLP annotation tool/library supports active learning

There is a tool Acharya which does this, available here (https://github.com/astutic/Acharya). You can upload your dataset and then add any algo to train a custom model and use it for further ...
2 votes

how to build an images dataset efficiently

There are a couple of different options to solve this problem: Create a custom Dataloader class - Depending on the modeling framework, different file formats can be read and modified at the loading ...
2 votes
Accepted

how to build an images dataset efficiently

Yes, You can use Image processing tools, Such as ImageMagick[1], GraphicsMagick[2](Similar to ImageMagick), and PIL[3] (Python Imaging Library or Known as Pillow). These tools can convert images ...
  • 202
0 votes

When is the right moment to split the dataset?

It's better to split the data into training and testing sets before doing things like scaling and imputation. This is because these steps are usually done using parameters learned from the training ...
1 vote

When is the right moment to split the dataset?

In principle, you can do many preprocessing activities (e.g., converting data types, removing NaN values, etc.) on the entire dataset since it does not make a difference whether these steps are ...
  • 126
1 vote

I'm Looking for dataset with Name, Father's Name, Mother's name, Age, Gender, Ethnicity, ~Birthplace

That's the kind of sensitive information that's hard to get and hard to protect, as defined at the University of Pittsburgh as personally identifiable information (PII). I personally recommend reading ...

Top 50 recent answers are included