New answers tagged

1

How you do this will depend a lot on the tools you are already using and the way your model trains. If you are iterating e.g. over a grid of parameters to search, it can be trivial to parallelize. Other setups can be quite difficult. There are some generic ways to do it using Python: check out the relevant documentation. One thing to be aware of is that ...


1

If you're using scikit-learn, there is a parameter for most learners called n_jobs. This parameter can be set to -1 to utilize all processors. For more details, see scikit-learn n_jobs parameter on CPU usage & memory


4

A way to speed up this process is to preprocess the large dataset, the goal being to store the documents from A in a way which avoids a lot of useless comparisons. Store each document from A in an inverted index $m$, so that for any word $w$ $m[w]$ is the list of all documents in A which contain word $w$ (note that a document can appear several times in ...


0

Are you trying to search for a specific date in the entire historical dataset? df[df['date_column'].isin('specified_date')]


2

Any good hash will be uniformly distributed, which means that you can assume a uniform distribution when you apply modulo n, as long as $n < 2^{M/2}$, where M is the number of bits in your hash, see here. So for SHA1-32 you would at most modulo by $2^{16}$. There is no approach to calculating an integer value; what you have there is an hexadecimal ...


Top 50 recent answers are included