Introduction
The first use case of the SPATIAL project leverages Federated Learning (FL) to build AI models using a distributed approach. The main benefit of using FL remains in removing the need to transfer training data to a central server. Instead, FL clients train locally a version of the AI model using their local datasets. Then, the models are sent to a central server that aggregates all the client updates. FL reduces the communication overhead while increasing privacy protection by default.
Nowadays, FL is seen as a promising computing paradigm that could allow training the AI models of the future by leveraging distributed computing capacities, such as those offered by existing communication networks.
Advantages
FLaaS demonstrates several key advantages with respect to classic centralized ML training:
- Enhanced Privacy and Security: Data never leaves the local devices, minimizing the risk of exposure and ensuring higher compliance with privacy laws.
- Reduced Latency and Bandwidth Usage: By avoiding the need to continuously transfer large datasets to a central server, FL reduces the demand on network bandwidth and lowers latency, making it feasible for deployment in edge computing environments.
- Scalability: FL can easily scale to accommodate a large number of clients, each contributing to the global model without the need for a powerful centralized infrastructure.
- Robustness and Fault Tolerance: Decentralized training can improve the robustness of the learning process, as the system does not rely on a single point of failure. Even if some clients go offline, the training can continue with the remaining active clients.
FLaaS (Federated Learning as a Service) builds on these foundational principles to offer a comprehensive framework that simplifies the deployment and management of federated learning systems. FLaaS aims to provide a seamless experience for organizations looking to implement FL, offering tools and services that handle the complexities of client coordination, model aggregation, and communication protocols.
Implementation
In this context, Telefónica has developed FLaaS, the Federated Learning as a Service platform that attempts to facilitate practical FL on Android devices and can be extended to support new types of end devices. Similar to Machine Learning as a Service (MLaaS) platforms, FLaaS reduces the learning curve and configuration required to run FL projects.
To achieve that, as depicted in the figure above, FLaaS offers a number of components and functionalities:
- Admin interface: Offering a user-interface for the AI developer and System administrator to configure, run and monitor the execution of FL projects.
- FLaaS server: The backend of the service performs the orchestration and aggregation required for the FL process.
- FLaaS local: Library available for Android devices that facilitates the training of models combining the data of multiple applications installed on the device using two different approaches: (i) combining their data (Joint Samples, JS); (ii) combining their models trained individually (Joint Models, JM)
- FLaaS communication service: This service enables a push-based communication channel between the server and the clients, which is required for FL tasks such as starting the training process.
Despite not requiring the sharing of user data, FL does not offer complete privacy guarantees. Previous research has shown how successful privacy attacks can still be crafted against models even when trained using FL. To address this problem, thanks to the integration between FLaaS and the SPATIAL Differential Privacy (DP) component available in the SPATIAL platform, we have incorporated a new privacy-related feature in FLaaS. It corresponds to the option of adding Central Differential Privacy (CDP) to the FL process. To include CDP, the admin has the option of configuring and parametrizing the level of DP that is added to the aggregation process. By including CDP, we add an additional level of privacy protection to safeguard the privacy of the users participating in any FL project.
Our results have shown how the inclusion of CDP increases the resilience of the trained models against privacy attacks, reducing the capacity of a skilled attacker to retrieve sensitive information from them.
Potential Applications
FLaaS can be applied across various sectors where data privacy is paramount and distributed data sources are abundant:
- Healthcare: Training predictive models using patient data from multiple hospitals without compromising patient confidentiality.
- Finance: Developing fraud detection algorithms by combining insights from different financial institutions without sharing sensitive transaction data.
- Smart Cities: Enhancing urban planning and management by analyzing data from distributed IoT devices without aggregating all data centrally.
Evaluation
We have evaluated FLaaS through rigorous testing in both controlled in-lab environments and real-world user studies to ensure its robustness, usability, and effectiveness.
Figure 1 Screenshot of the FLaaS demo app running on an Android device
In-Lab Testing
Our in-lab testing focused on validating the core functionalities of FLaaS, including model training, aggregation, and deployment. We conducted extensive performance benchmarking to assess the aggregation server’s efficiency and the client SDK’s scalability across various hardware configurations. Security assessments were performed to ensure that the privacy-preserving mechanisms, such as differential privacy and secure multiparty computation, functioned correctly and did not introduce significant performance bottlenecks.
User Studies
End-User Evaluation
To understand the impact of FLaaS on end-users, we conducted a series of field studies involving diverse participants using different types of devices. These studies focused on two main aspects:
- Device Performance: We monitored the effect of running FLaaS on device performance metrics such as CPU usage, memory consumption, battery life, and network bandwidth. Our goal was to ensure that FLaaS operates efficiently without significantly degrading the performance or usability of end-user devices.
- User Perception: Participants were asked to provide feedback on their experiences with FLaaS through structured interviews and questionnaires. This feedback helped us understand the perceived impact of FLaaS on their device performance and their comfort level with the federated learning process. The results indicated a generally positive reception, with most users appreciating the privacy benefits and minimal impact on device performance.
AI Practitioner Survey
We also conducted a survey with 20 AI practitioners to evaluate the backend and administrative interface of FLaaS. This survey aimed to gather insights on the usability, functionality, and overall satisfaction with the system from those who would be managing and deploying FLaaS in real-world scenarios. Key aspects evaluated included:
- Ease of Use: Practitioners assessed the intuitive and user-friendly nature of the admin interface, focusing on the learning curve and ease of integration with existing workflows.
- Functionality: The survey evaluated the range of features available in the backend, such as monitoring tools, model management capabilities, and security configurations.
- Performance and Reliability: Practitioners provided feedback on the system’s performance, reliability, and scalability when managing multiple federated learning clients and large datasets.
The survey results were overwhelmingly positive, with practitioners highlighting FLaaS’s key strengths as seamless integration, comprehensive monitoring tools, and robust security features.
Sociotechnical Analysis
Finally, we performed a comprehensive sociotechnical analysis to evaluate the broader impact of FLaaS within organizational and social contexts. This analysis was performed using COMPASS, an algorithmic assessment framework developed within the SPATIAL project that considers various factors, including Context, Accountability, Measures utilized, and Privacy potentials.
The sociotechnical analysis confirmed that FLaaS not only meets technical requirements but also aligns well with ethical standards and organizational goals, highlighting its inherent focus on privacy protection and the need for continuous stakeholder engagement to maintain trust and acceptance.
Conclusion
In conclusion, our comprehensive evaluation of FLaaS through in-lab testing, user studies, AI practitioner surveys, and sociotechnical analysis has demonstrated its effectiveness and viability as a federated learning framework. These evaluations have provided valuable insights that have guided the refinement of FLaaS, ensuring it is a robust, user-friendly, and privacy-preserving solution for modern AI model training.
Watch the video! https://youtu.be/uQ-sSBBSlU4