Modeling Decisions for Artificial Intelligence by Unknown

Author:Unknown
Language: eng
Format: epub
ISBN: 9783030575243
Publisher: Springer International Publishing

4 Experimental Results

Experiments have been carried out on two public data sets: MNIST and CIFAR-10. MNIST consists of 70, 000 images of handwritten digits (which correspond to classes), from 0 to 9. Each image is gray-scaled and pixels in size. We used 60, 000 of them for training and the remaining 10, 000 for testing. The CIFAR-10 dataset contains of 60, 000 colored images of 10 classes. Each class has 6, 000 images. We used 50, 000 images for training and the remaining 10, 000 for testing. For both datasets, we divided the training data set into 100 equally sized shards, which were distributed among 100 simulated workers.

For MNIST, we used a small DL model with 21,840 trainable parameters. The earlier layers were convolutional layers that extracted the features and passed them to the last two fully connected layers: fully connected layer 1 (FC1), and fully connected layer 2 (FC2). FC1 contained 50 neurons with 1 bias for each neuron, while FC2 contained 10 neurons with 1 bias for each neuron. The output of FC2 was fed to a SoftMax layer that introduced the final probabilities of the 10 classes. For CIFAR-10, we used a large DL model that contained 14,728,266 trainable parameters in total. The earlier layers extracted the features and they had only one fully connected layer (FC1) that contained 10 neurons with 1 bias for each neuron. The output of the fully connected layer was fed to a SoftMax layer that introduced the final probabilities of the 10 classes.

We trained the small DL model for 30 global rounds. Each worker trained the model locally using the stochastic gradient descent (SGD) with 5 local epochs, local batch size = 20, learning rate = 0.001, and momentum = 0.9. We trained the large DL model for 60 global rounds. Each worker trained the model locally using the stochastic gradient descent (SGD) with 5 local epochs, local batch size = 50, learning rate = 0.01 and momentum = 0.9.

For both models, we used the precision as a performance metric. Precision is the amount of true positives TP divided by the sum of true positives and false positives FP, that is .

In the experiments the server selected a fraction of workers at each round. We limited the number of malicious updates per round between 1 to 6. This is consistent with the Byzantine resilience property, which states that a method is considered to be Byzantine resilient if it is robust against Byzantine workers where [1].

Figure 3 shows the evolution of the classification precision achieved by the small DL model at each round when no detection is performed. The dashed line shows the precision when all the workers acted honestly. The global model achieves a precision around after 30 rounds. In addition to the all-honest case, we considered different scenarios where a fraction of the workers were malicious. In our experiments, malicious workers trained their model honestly and, after that, they added some random noise to each parameter in their model from a Gaussian distribution with mean 0 and standard deviation 0.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.