I'm looking for examples of Machine Learning / Neural Networks examples that work with quantized weights, activation functions,.... The simple approach of training with floating point parameters and then quantizing does not give results. I'm working with pytorch now but can use other packages if necessary.
5
-
$\begingroup$ It does not work? It is all quantized. Many learners can work on 16-bit representations. Gradient-descent has to be performed on the discretized parameters. (ieeexplore.ieee.org/document/8907458) $\endgroup$– EngrStudentJun 23 at 3:39
-
$\begingroup$ I'm working with much less than 16 bits...here's a survey of methods arxiv.org/abs/2103.13630 I'm looking for coded examples $\endgroup$– unknownJun 23 at 15:18
-
$\begingroup$ So, like this one? ibm.com/blogs/research/2018/12/8-bit-precision-training $\endgroup$– EngrStudentJun 23 at 15:55
-
1$\begingroup$ that's the general idea...some starter code would help me get up to speed $\endgroup$– unknownJun 23 at 15:59
-
1$\begingroup$ So, like this: github.com/MatthieuCourbariaux/8-bit-deep-learning ? Seems to have "QuantizedDenseLayer" in the python code. $\endgroup$– EngrStudentJun 23 at 16:05
Add a comment
|