Clojure for Data Science by Henry Garner

Author:Henry Garner
Language: eng
Format: epub
Publisher: Packt Publishing

We could implement SGD with Tesser by taking advantage of the fact that the combiner is applied serially, and treat each chunk as a mini-batch from which the coefficients could be calculated. This would mean that our reduce step was the identity function—we have no reduction to perform.

Instead, let's use this as an opportunity to learn more on how to construct a Hadoop job in Parkour. Before delving more into Parkour, let's see how stochastic gradient descent could be implemented using what we already know:

(defn stochastic-gradient-descent [options data] (let [batches (->> (into [] data) (shuffle) (partition 250)) descend (fn [coefs batch] (->> (gradient-descent-fold (assoc options :coefs coefs)) (t/tesser (chunks batch))))] (reductions descend (:coefs options) batches)))

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.