Hot answers tagged

3

If I understood correctly, you will eventually have, after some time, those user ratings right? So, assuming that you will have some labeled data (i.e. user ratings together with the features you say) to train with, you can build a multivariate regression model (you can have a first look at linear models to begin with). This approach is similar to what you ...


2

The reasoning will be: "The more data for training the better". Then you have to keep in mind that the validation/hold-out set has to resemble how it should work on production/testing. The theory is that the larger the training data, the better the model should generalize. The validation set can be much smaller, on extremely big dataset you can ...


1

I think you are trying to change all the values in certain column to abspath you can simply use apply function def full_path(x): return "my path" + "specific values" + ".jpg" df[list(df.keys())[0]]=df[list(df.keys())[0]].apply(full_path) If not I have one more solution you can store the values which will be the replacements ...


1

I don't know many boosting packages but I've been using XGboost for a while now and the biggest tabular dataset I've had was more than 40 times smaller than yours. The training took 2-3 days. In my experience training time is worse than linear with the size of the data even though it highly depends on the data itself and the hyperparameters you chose. My ...


1

Quite often with massive datasets the model doesn't actually need the whole data. So I think the first step is to check whether using the whole data is useful: run an ablation study where you use say 1%, then 2%, 3%, .., up to say 10% of the data (adapt the levels to your case of course). Each run consists in training on the x % subset and evaluating on a ...


1

The variation in performance gains you are seeing from reduction precision might be do to different frameworks using different types. Even after downcasting data types, some operations will automatically upcast types. You mention using Pandas and TensorFlow / Keras. Mixing these frameworks leads to unwanted recasting of data types. It is better to use a ...


1

Strictly theoretically it makes no difference on DNN, I answered it today here and I said: Here is why: We already know mathematically that NN can approximate any function. So lets say that we have Input X. X is highly correlated, than we can apply a decorrelation technique out there. Main Thing is, you get X` that has different numerical representation. ...


1

We built the symptom checker with AI API endlessmedical. We provide free to download for any use source code of entire symptom checker here: https://endlessmedical.com/chatbot-like-symptom-checker-free-source-code-download/ Have fun!


1

One option is fingerprinting. If two objects have the same fingerprint, they are probably the same object. Depending the technique used, the fingerprint can not tell about approximate duplicates.


1

What is the purpose of the analysis? What are the criteria of primary interest (accuracy)? The class imbalance problem stems from having insufficient data from the minority class to adequately characterise it's distribution. This means imbalance is only a problem if you have a small dataset, if you have lots of data, the imbalance problem generally ...


1

There is a design pattern call "strangler" that might be applicable. The strangler design pattern leaves all legacy systems in place and migrates piece-by-piece to a single, updated system. It does this by creating a proxy interface that routes requests to either legacy system or the updated system. As the migration happens, the proxy routes more ...


Only top voted, non community-wiki answers of a minimum length are eligible