Deep In-memory Architectures for Machine Learning by Mingu Kang & Sujan Gonugondla & Naresh R. Shanbhag

Author:Mingu Kang & Sujan Gonugondla & Naresh R. Shanbhag
Language: eng
Format: epub
ISBN: 9783030359713
Publisher: Springer International Publishing

(4.14)

where B x is the precision of X, which is fixed at 8-b in this application. The precision B Δ = 16 is set as B x = 8 and batch sizes of up to N = 256 were to be accommodated.

It can be shown [73] that a necessary condition for convergence (stopping criterion) of the SGD update in (4.11) is given by,

(4.15)

The precision B WUD = 16 is set as algorithmic simulations indicate that the algorithm converges with a learning rate γ ≥ 2−15. In addition, the batch-mode algorithm offers an interesting trade-off between the batch size N and the learning rate γ, which will be studied in Sect. 4.3.2.

The regularization factor λ has an optimum value that lies in between an upper bound that constrains the magnitude of W and a lower bound needed to avoid overflow in the weight accumulator (4.11). It can be shown that a sufficient condition to prevent overflow in the weight accumulator is:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.