Federated Learning at the Edge Can Beat the Cloud in Privacy, Speed, and Cost

In the 2000s, the “cloud” began to take off. Programmers and companies began purchasing on-demand virtual computing resources to run their software and applications.

Over the past two decades, developers have become accustomed to and dependent on instantly available infrastructure managed and maintained by someone else. And that’s no surprise. The abstraction of hardware and infrastructure allows developers and companies to focus on product innovation and user capabilities above all else.

Amazon Web Services, Microsoft Azure and Google Cloud have made storage and compute ubiquitous, on-demand and easy to deploy. And these hyperscalers have built robust, high-margin businesses on this approach. Cloud-dependent organizations have traded capital expenditures (servers and hardware) for operating expenses (pay-as-you-go compute and storage resources).

Enter federated learning

While the ease of use of the cloud is a boon for any start-up team trying to innovate at all costs, the cloud-centric architecture is a significant revenue cost as the company grows. In fact, 50% of revenue for large SaaS companies goes into cloud infrastructure.

As machine learning (ML) continues to grow in popularity and utility, organizations are storing increasing amounts of data in the cloud and training ever larger models in pursuit of greater model accuracy and greater user benefit. This further increases dependency on cloud providers and organizations find it difficult to repatriate workloads to on-premises solutions. In fact, it would require them to hire a stellar infrastructure team and completely re-engineer their systems.

Organizations are looking for tools that enable new product innovation and deliver high accuracy with low latency while being cost-effective.

Enter federated learning (FL) at the edge.

What is Federated Learning (FL) at the edge?

FL, or collaborative learning, takes a different approach to storing and computing data. For example, while popular cloud-centric approaches to ML send data from your phone to centralized servers and aggregate that data in a silo, FL at the edge keeps the data on the device (i.e., your phone or tablet). It works as follows:

Step 1: Your edge device (or cell phone) downloads an initial model from an FL server.

Step 2: On-device training is then conducted; the data on the device improves the model.

Step 3: Encrypted training results are sent back to the server for model improvement, while the underlying data remains securely on the user’s device.

Step 4: With the model on the device, you can perform training and inference at the edge in a fully distributed and decentralized way.

This loop continues iteratively and the accuracy of the model increases.

User benefits of federated learning

When you are not dependent on or constrained by centralized data, the user benefits in dramatic ways. With FL capped, developers can improve latency, reduce network calls, and increase power efficiency, while promoting user privacy and improving model accuracy.

FL on the edge is enabled by the ever-increasing hardware capacity of the phones in our pockets. Every year, on-device computing and battery life improve. As the processor and hardware of the smartphone in our pocket improves, FL techniques will reveal increasingly complex and personalized use cases.

Imagine, for example, software that sits on your phone in a privacy-centric manner that can automatically compose replies to incoming emails with your individual tone, punctuation style, slang and other hyper-personalized attributes – all you have to do is click send.

The business attraction is strong

In my conversations with several Fortune 500 companies, it became extremely obvious how much demand there is for FL at the edge across all industries. CTOs express how they are looking for a solution to bring FL techniques to life at the edge. CFOs reference the millions of dollars spent on infrastructure and model deployment that could otherwise be saved in an FL approach.

In my opinion, the three industries with the greatest potential to reap the rewards of federated learning are finance, media, and e-commerce. Let me explain why.

Use case #1: Finance — improved latency and security

Many large multinational financial companies (Mastercard, PayPal) are eager to adopt FL to the edge to help them identify account takeovers, money laundering and fraud detection. More accurate models are sitting on the shelf and have not been approved for release.

Why? These models increase latency just enough that the user experience is negatively impacted – we can all think of apps that we no longer use because they took too long to open or crashed. Companies cannot lose users for these reasons.

Instead, they accept a higher false negative rate and suffer excessive account hijacking, laundering and fraud. FL on the edge allows companies to simultaneously improve latency while showing a relative increase in model performance compared to traditional cloud-centric deployments.

In the media sector, companies like Netflix and YouTube want to increase the relevance of their suggestions about which movies or videos to watch. The Netflix Prize awarded $1 million for a 10% increase in performance compared to its own algorithm.

FL on the Edge has the potential to deliver a similar impact. Today, when a new show comes out or a popular sporting event is broadcast (like the Superbowl), companies reduce the signals they pick up from their users.

Otherwise, the sheer volume of data (at a rate of millions of requests per second) causes a network bottleneck that prevents them from recommending content at scale. With edge computing, companies can leverage these signals to suggest personalized content based on insight into individual user tastes and preferences.

Use Case #3: E-Commerce — More Timely and Relevant Suggestions

Ultimately, e-commerce and marketplace companies want to increase click-through rates (CTR) and drive conversions based on real-time resource stores. This allows them to reclassify recommendations for customers and provide more accurate forecasts without the lag of traditional cloud-based recommendations.

Imagine, for example, opening the Target app on your phone and getting highly personalized product recommendations in a completely privacy-centric way – no identifying data would have left your phone. Federated learning can increase CTR thanks to a more efficient, privacy-aware model that provides users with more timely and relevant suggestions.

The market scenario

Thanks to technological advances, large corporations and start-ups are working to make FL more ubiquitous so that businesses and consumers alike can benefit. For companies, this likely means lower costs; for consumers, it can mean a better user experience.

There are already some early players in the space: Amazon SageMaker allows developers to deploy ML models primarily on edge devices and embedded systems; Google Distributed Cloud extends your infrastructure to the edge; and Nimbleedge start-ups are reinventing the infrastructure stack.

While we’re in the first innings, FL is on edge and the hyperscalers are in a starter’s dilemma. The revenue that cloud providers earn for compute, storage and data is at risk; Modern vendors that have embraced edge computing architecture can offer customers premium ML model accuracy and reduced latency. This improves the user experience and drives profitability — a value proposition you can’t ignore for too long.

Neeraj Hablani is a partner at Neotribe Ventures focused on early stage companies that create innovative technologies.