El constructivismo en la enseñanza de las ciencias

Information is instinctive at the edge with trillions of smart-phones and other gadgets continually generating data; this data generated can enable improved products and keener models. On-device inference offers an improvement to latency, enable works offline, often has a battery life advantages and can also have privacy advantages because a server does not need to be in the loop for every communication have with that locally generated data. Federated learning trains the information distributed on mobile gadgets and learns as a distributed model by aggregating locally calculated updates through a central synchronising server.

A significant benefit of this methodology is the decoupling of model learning from the need for straight admission to the raw learning information. Approximately expectation of the server synchronising the training is still obligatory and contingent on the particulars of the model and algorithm, and the updates may still contain private information. However, for purposes where the training objective can be detailed based on information accessible on each client, federated learning can meaningfully reduce privacy and security risks by

controlling only the machine, rather than the device and the on-demand computing system. If extra privacy is needed, randomisation methods from differential privacy used. The

centralised algorithm could be changed to give a differentially confidential model which acknowledges the model to be published while protecting the confidentiality of the entities influencing updates to the training process.

Let us consider a disease analysis scheme as an illustration. A collection of medical facilities wants to develop a joined system for disease analysis. However, apiece medical centre with a dissimilar patient as well as a dissimilar kind of health examination outcome. Transfer learning [39] is a conceivable answer for such circumstances. Also, [40] suggested protected federated learning systems which can learn a depiction among the characteristics of the organisation using a typical instance.

34

Conclusions

The segregation of data and the importance of information confidentiality are seemly the next tasks for non-natural intelligence, but federated learning has familiarised new anticipation. Federated learning could create an untied prototype for numerous establishments while the local information is secured so that the organisation could win together taking the

information confidence principle.

The thesis, by and large, presents the fundamental theories, style and procedures of federated learning and its hypothetical in the diverse application. Privacy interests are the fundamental enthusiasm behind this approach to machine learning. Data from consumer devices are private and should not transfer to a server. Federated learning permits the training a model on the data from a consumer’s devices by processing it locally. Only weight updates derived using the data sent to a server.

The optimisation process can compute unbiased gradient estimates, like mini-batch gradient descent. Besides, it can take a long period until a single iteration completed since the server needs to wait until users can respond with the updates to reduce convergence time, several optimisation-related strategies familiarised.

Straight forward implementation of federated learning can require much communication between customers and the server, applying a federated learning scale efficiently with the size of the model, special compression techniques are studied.

The implementation of federated learning, ensuring that privacy is guaranteed, differential privacy methods were studied. During the process, it realised that the speed of download a data is critical in privacy. Also, by bounding how much an individual can influence the model weights and by randomising updates, can quantify how difficult it is to arrive at an

assumption about the individual.

Besides, other strategies for personalising models in federated learning was studies like the method based on transfer learning which enables the customisation of models locally.

Furthermore, differential private learning has a theoretical guarantee for the level of privacy; the computational cost is enormous; hence future research could focus on making it more feasible to implement the technique.

35

An area of research that studied in future is implementing cryptographic encryption methods; it can ensure that the server can only read updates from users once several the update is received hence avoiding man-in-the-middle attack.

Nevertheless, it expected that in future federated learning can assist in the breaking of the barriers among organisations and develop a society where data and information distributed with safety and the advantages equally and distributed based on the contribution of each member.

36

Reference

[1] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, B. He, ‘Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection’, ArXiv190709693 Cs Stat, Oct. 2019. [2] A. P. Sheth and J. A. Larson, ‘Federated database systems for managing distributed,

heterogeneous, and autonomous databases’, ACM Computer Survey., vol. 22, no. 3, pp. 183–236, Sep. 1990, DOI: 10.1145/96602.96604.

[3] D. Liu, ‘Design and Analysis of an Interoperable HLA-based Simulation System over a Cloud Environment’, p. 110.

[4] T. Yang et al., ‘Applied Federated Learning: Improving Google Keyboard Query Suggestions’, ArXiv181202903 Cs Stat, Dec. 2018.

[5] J. P. Albrecht, ‘How the GDPR Will Change the World’, 2016, DOI: 10.21552/edpl/2016/3/4.

[6] I. Giacomelli, S. Jha, C. D. Page, and K. Yoon, ‘Privacy-Preserving Ridge Regression on Distributed Data’, p. 25.

[7] K. Cheng, T. Fan, Y. Jin, Y. Liu, T. Chen, and Q. Yang, ‘Secure Boost: A Lossless Federated Learning Framework’, ArXiv190108755 Cs Stat, Jan. 2019.

[8] V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, ‘Federated Multi-Task Learning’,

ArXiv170510467 Cs Stat, Feb. 2018.

[9] L. Zhao et al., ‘InPrivate Digging: Enabling Tree-based Distributed Data Mining with Differential Privacy’, in IEEE INFOCOM 2018 - IEEE Conference on Computer

Communications, Honolulu, HI, 2018, pp. 2087–2095, DOI:

10.1109/INFOCOM.2018.8486352.

[10] S. Hardy et al., ‘Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption’, ArXiv171110677 Cs, Nov. 2017. [11] ‘NWIJBT13.pdf’, Google Docs. [Online]. Available:

https://docs.google.com/file/d/0B5qjSJpwjTbNYmRUVjNkZHVyaWs/edit?usp=sharing &usp=embed_facebook. [Accessed: 17-Nov-2019].

[12] Q. Yang, Y. Liu, T. Chen, and Y. Tong, ‘Federated Machine Learning: Concept and Applications’, ArXiv190204885 Cs, Feb. 2019.

[13] K. Bonawitz et al., ‘Towards Federated Learning at Scale: System Design’,

ArXiv190201046 Cs Stat, Mar. 2019.

37

[15] A. Graves, A. Mohamed, and G. Hinton, ‘Speech recognition with deep recurrent neural networks’, IEEE International Conference on Speech, Signal and Acoustics

Processing, Vancouver, BC, Canada, 2013, pp. 6645–6649, DOI:

10.1109/ICASSP.2013.6638947. [16] ‘NetflixPrize-description.pdf’.

[17] A. Narayanan and V. Shmatikov, ‘How to Break Anonymity of the Netflix Prize Dataset’, arXiv:cs/0610105, Nov. 2007.

[18] I. Wagner and D. Eckhoff, ‘Technical Privacy Metrics: A Systematic Survey’, ACM

Comput. Surv., vol. 51, no. 3, pp. 1–38, Jun. 2018, DOI: 10.1145/3168389.

[19] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, ‘Communication-Efficient Learning of Deep Networks from Decentralized Data’,

ArXiv160205629 Cs, Feb. 2017.

[20] N. Papernot, M. Abadi, Ú. Erlingsson, I. Goodfellow, and K. Talwar, ‘Semi- supervised Knowledge Transfer for Deep Learning from Private Training Data’,

ArXiv161005755 Cs Stat, Mar. 2017.

[21] M. Yurochkin, ‘Poster #20 Bayesian Nonparametric Federated Learning of Neural Networks’, p. 9.

[22] K. Bonawitz et al., ‘Practical Secure Aggregation for Privacy-Preserving Machine Learning’, in Proceedings of the 2017 ACM SIGSAC Conference on Computer and

Communications Security - CCS ’17, Dallas, Texas, USA, 2017, pp. 1175–1191, DOI:

10.1145/3133956.3133982.

[23] O. Goldreich, ‘Secure Multi-Party Computation’, p. 110.

[24] C. Dwork and A. Roth, ‘The Algorithmic Foundations of Differential Privacy’,

Found. Trends® Theor. Comput. Sci., vol. 9, no. 3–4, pp. 211–407, 2013, DOI:

10.1561/0400000042.

[25] R. C. Geyer, T. Klein, and M. Nabi, ‘Differentially Private Federated Learning: A Client Level Perspective’, ArXiv171207557 Cs Stat, Dec. 2017.

[26] Ú. Erlingsson, V. Pihur, and A. Korolova, ‘RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response’, in Proceedings of the 2014 ACM SIGSAC

Conference on Computer and Communications Security - CCS ’14, Scottsdale, Arizona,

USA, 2014, pp. 1054–1067, DOI: 10.1145/2660267.2660348.

[27] M. Abadi et al., ‘Deep Learning with Differential Privacy’, Proc. 2016 ACM SIGSAC

Conf. Comput. Commun. Security. - CCS16, pp. 308–318, 2016, DOI:

38

[28] H. B. McMahan, L. Zhang, D. Ramage, and K. Talwar, ‘LEARNING

DIFFERENTIALLY PRIVATE RECURRENT LANGUAGE MODELS’, p. 14, 2018. [29] R. Pascanu, T. Mikolov, and Y. Bengio, ‘On the difficulty of training Recurrent

Neural Networks’, ArXiv12115063 Cs, Feb. 2013.

[30] ‘RECURRENT NEURAL NETWORKS’, p. 389, 2001.

[31] C. M. Bishop, ‘Training with Noise is Equivalent to Tikhonov Regularization’,

Neural Comput., vol. 7, no. 1, pp. 108–116, Jan. 1995, DOI: 10.1162/neco.1995.7.1.108.

[32] D. Winograd-Cort, A. Haeberlen, A. Roth, and B. C. Pierce, ‘A framework for adaptive differential privacy’, Proc. ACM Program. Lang., vol. 1, no. ICFP, pp. 1–29, Aug. 2017, DOI: 10.1145/3110254.

[33] C. Clifton and T. Tassa, ‘On syntactic anonymity and differential privacy’, in 2013

IEEE 29th International Conference in Data Engineering Workshops (ICDEW),

Brisbane, QLD, 2013, pp. 88–93, DOI: 10.1109/ICDEW.2013.6547433.

[34] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, ‘Exploiting Unintended Feature Leakage in Collaborative Learning’, ArXiv180504049 Cs, May 2018.

[35] Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally, ‘Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training’, ArXiv171201887 Cs Stat, Dec. 2017.

[36] L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, ‘Privacy-Preserving Deep Learning via Additively Homomorphic Encryption’, 715, 2017.

[37] D. Leroy, A. Coucke, T. Lavril, T. Gisselbrecht, and J. Dureau, ‘Federated Learning for Keyword Spotting’, ArXiv181005512 Cs Eess Stat, Feb. 2019.

[38] B. Hitaj, G. Ateniese, and F. Perez-Cruz, ‘Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning’, ArXiv170207464 Cs Stat, Feb. 2017. [39] S. J. Pan and Q. Yang, ‘A Survey on Transfer Learning’, IEEE Trans. Knowl. Data

Eng., vol. 22, no. 10, pp. 1345–1359, Oct. 2010, DOI: 10.1109/TKDE.2009.191. [40] Y. Liu, T. Chen, and Q. Yang, ‘Secure Federated Transfer Learning’,

ArXiv181203337 Cs Stat, Dec. 2018.

[41] Z. Zheng, S. Xie, H.-N. Dai, X. Chen, and H. Wang, ‘Blockchain challenges and opportunities: a survey’, p. 24.

[42] G. Liang and S. S. Chawathe, ‘Privacy-Preserving Inter-database Operations’, in

Intelligence and Security Informatics, 2004, pp. 66–82.

[43] V. Nikolaenko, U. Weinsberg, S. Ioannidis, M. Joye, D. Boneh, and N. Taft, ‘Privacy- Preserving Ridge Regression on Hundreds of Millions of Records’, in 2013 IEEE

39

Symposium on Security and Privacy, Berkeley, CA, 2013, pp. 334–348, DOI:

10.1109/SP.2013.30.

[44] Y. Zhang and Q. Yang, ‘A Survey on Multi-Task Learning’, ArXiv170708114 Cs, Jul. 2018.

[45] P. Blanchard, E. M. E. Mhamdi, R. Guerraoui, and J. Stainer, ‘Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent’, p. 11.

[46] S. Truex et al., ‘A Hybrid Approach to Privacy-Preserving Federated Learning’,

ArXiv181203224 Cs Stat, Aug. 2019.

[47] P. Paillier, ‘Public-key cryptosystems based on composite degree residuality classes’,

in In Advances in Cryptology — Eurocrypt 1999, 1999, pp. 223–238.

[48] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, and M. Jirstrand, ‘A Performance Evaluation of Federated Learning Algorithms’, in Proceedings of the Second Workshop

on Distributed Infrastructures for Deep Learning - DIDL ’18, Rennes, France, 2018, pp.

1–8, DOI: 10.1145/3286490.3286559.

[49] J. Konecˇný, H. B. McMahan, F. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, ‘Federated Learning: Strategies for Improving Communication Efficiency’, p. 5. [50] ‘Wang_Yushi_201709_MSc.pdf’.

[51] S. Caldas et al., ‘LEAF: A Benchmark for Federated Settings’, ArXiv181201097 Cs Stat, Jan. 2019.

[52] X. Wang, Y. Han, C. Wang, Q. Zhao, X. Chen, M. Chen, ‘In-Edge AI:

Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning’, ArXiv180907857 Cs, Jul. 2019.

[53] G. Ulm, E. Gustavsson, and M. Jirstrand, ‘Functional Federated Learning in Erlang (ffl-erl)’, ArXiv180808143 Cs, vol. 11285, pp. 162–178, 2019, DOI: 10.1007/978-3-030- 16202-3_10.

[54] A. Hard et al., ‘Federated Learning for Mobile Keyboard Prediction’,

ArXiv181103604 Cs, Feb. 2019.

[55] T. Nishio and R. Yonetani, ‘Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge’, ArXiv180408333 Cs, Oct. 2018. [56] S. Samarakoon, M. Bennis, W. Saad, and M. Debbah, ‘Distributed Federated

Learning for Ultra-Reliable Low-Latency Vehicular Communications’, ArXiv180708127

Cs Math, Aug. 2018.

[57] I. Damgard, V. Pastro, N. P. Smart, and S. Zakarias, ‘Multiparty Computation from Somewhat Homomorphic Encryption’, 535, 2011.

40

[58] F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, ‘Robust and Communication- Efficient Federated Learning from Non-IID Data’, ArXiv190302891 Cs Stat, Mar. 2019. [59] H. Zhu and Y. Jin, ‘Multi-objective Evolutionary Federated Learning’,

ArXiv181207478 Cs Stat, Jun. 2019.

[60] E. Jeong, S. Oh, H. Kim, J. Park, M. Bennis and S.-L. Kim, ‘Communication- Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data’, ArXiv181111479 Cs Stat, Nov. 2018.