# Using sklearn DictVectorizer in real-time systems

Any binary one-hot encoding is aware of only values seen in training, so features not encountered during fitting will be silently ignored. For real time, where you have millions of records in a second, and features have very high cardinality, you need to keep your hasher/mapper updated with the data.

How can we do an incremental update to the hasher (rather calculating the entire fit() every time we incounter a new feature-value pair)? What is the suggested approach here the tackle this?

D = [{'foo': 1, 'bar': 2}, {'foo': 3, 'baz': 1}] 
Then you don't need to update the hash every time, all you have to do is transform D into the vectorized format and add it to your collection. As long as you don't need to fit new features this can be done relatively fast. However, if you're doing something less structured like NLP this won't give you hardly any useful information.