(Image via BT)

The UK's largest telco, BT, is rethinking its approach to data use internally and externally, with the aim of becoming an event-driven organization. The company hopes that over time it will be able to expose real-time data events externally, in an attempt to monetize them and create new revenue streams. 

BT is doing this through the creation of an internal managed service, which is being led by Philip Parker, Principal Enterprise Integration Architect at BT, who is developing a real-time time data platform that is enabled and managed through efficient enterprise information management. This is being done via the use of Confluent's Kafka platform, which is connecting data across BT's on-premise infrastructure, as well as in the cloud with GCP and AWS. 

According to Parker, the need for a real-time events driven platform is critical, against the backdrop of a competitive marketplace. He said: 

Typically, the easier it is for a customer to move companies, the more they're disrupted. With telcos, it's pretty easy to move, so we are being really disrupted. Our typical revenue streams, where we get revenue from, they're actually going down. 

The biggest threat to telcos and our biggest issue is over the top services. Unlike water, where you turn on the tap and pay for what you use - you don't with the internet. You can stream Netflix all day long and you don't expect to pay any more to any of your broadband providers. We've then got things like WhatsApp and FaceTime taking away from our calls revenue. 

And if we look at the future. There are now 5G networks that you can use from satellites that are orbiting the earth, rather than from pylons. I want to propose a situation where Amazon Prime says: as part of Prime, you're going to get all your phone and data for free. What do BT do then? That's a huge worry. The picture I am trying to paint is that our revenue streams, I would say, are actually shrinking. We've got the biggest market share, so all we can do is try and keep it. 

To ensure that BT continues to excel in the future, Parker said that the organization needs to focus on two priorities. Firstly, the company needs to extract more value from its data. Secondly, it needs to prioritize real-time and better customer experiences. These two are also closely linked. 

He provided an example, whereby a customer experiences a broadband outage. The customer is then notified about the outage immediately and is provided with some extra data to their phone to tether a connection, whilst the outage persists. In the meantime, engineers are immediately sent out to fix the connection. All of this requires access to real-time data, but also improves the customer experience. 

An event-driven model

BT is thinking about events within the context of two camps. Firstly, microservices and an application Event Driven Architecture (EDA). This will improve BT's fault tolerance, agility and scalability, Parker said. The other camp is real time data, which will enhance the customer experience. 

Parker said that events are also complementary to other delivery/integration modes, such as ETL, data virtualization and data replication, but added that companies need to think about where real-time data can bring value. He said: 

If we think about ETL, data virtualization and data replication, they're still going to be around. I'm still going to have to put a lot of data into warehouses and I'm still going to do some manipulation on that data. Probably not in real-time, but to come up with some analytics. 

Does data have real-time value? When we talk about HR data or operational data, a lot of that data doesn't really have real time value. You can run a report at the end of each month, do I really need to keep updated all the time? Probably not. Why not? Because every time I do real-time stuff there's an overhead, it's a lot more expensive.

But there are examples of where this is important - we've got things like IoT and smart cities, healthcare devices, cyber threats, network monitoring, enhanced customer experience, and real-time data for marketing to increase sales. 

A three phased approach

BT's initial vision, prior to Parker coming on board, was ‘publish anywhere, subscribe anywhere, using a centrally managed, common technology'. However, there were loads of different siloed implementations across BT and teams couldn't get access to or share data, meaning value was being lost. 

This vision transitioned into becoming ‘a self-serve event platform', where customers (and by customers, this could be internal or external) could subscribe, publish, onboard, or automate BT data. But this left the problem of needing to understand the history of the data, the lineage and its quality. If BT is going to be consuming and sharing real-time data, it needs to know that it's using quality data. 

This has led to Parker's current strategy of Enterprise Information Management. He said: 

Enterprise information management at BT is almost about being a data marketplace, having all the data that I might need with all delivery modes available, being able to browse it, being able to understand the history of the lineage, and then being able to consume it, and then have a delivery mode that's applicable to me. This is our vision now. 

Parker outlined a number of strategic themes BT is following in order to become an event-driven business. These include: 

• Follow the API management model - BT should be following as much of an API management model as possible e.g. event portal, single language description, sharing of events to encourage reusability, self service, both internal and external events

• Monitoring, security and governance - Given EDA adds complexities - eventual transaction consistency, more challenging to debug - BT needs to ensure that its monitoring and governance frameworks are very mature. It must consider tools with business transaction monitoring and tracking, with visualisation. 

• Foster an event culture, use well defined data - Unlike APIs, events are not commonly understood, as the area is maturing. It requires a mind shift for both technical and business stakeholders. Therefore BT needs to formulate an approach to altering its culture to fully capitalize on events. Integration teams must work hand in hand with data teams to ensure the right data in the right format is flowing through the platform. 

• Common technology and platform across hybrid cloud - BT should start with the premise that a single platform made up of common technologies will fulfill its event streaming needs. It should anticipate that data sources and targets could be anywhere across hybrid cloud and beyond (GCP, AWS and on-premise). It should be centrally managed. 

• Flexibility and customer focused - As understanding of both BT's needs and event streaming as a capability matures, BT needs to ensure that it designs for flexibility. BT must ensure that it makes the platform and its capabilities focused on internal and external customer outcomes e.g. ease of use, speed. 

As noted above, prior to Parker coming on board, BT had some siloed event streaming projects taking place. It is now moving towards events as a service, as illustrated by the diagram below. The star indicates that BT is currently between connecting clouds and creating a common holistic platform. 

(Image sourced via Philip Parker’s presentation at Confluent London event)

The end goal, in terms of how Confluent is being used as the data in motion platform, integrating data across clouds and on-premise, is highlighted by the following diagram: 

(Image sourced via Philip Parker’s presentation at Confluent London event)

For an understanding of how Confluent and event streaming data can be used across a business, take a read of diginomica's interview with Confluent CEO Jay Kreps. But Parker notes that his team is now heavily focused on selling the possibility of becoming an events-driven organization, internally. He said: 

How do we sell what events can do internally? How do we make them realize? I think integration is terrible at this, I actually get vendors in to do it. But we at BT need to start and get much better. And then the big one for me…what's the big seller in API management? Well it's exposing APIs and monetising them, right? And that's what we're going to do with events. How do you capitalize all that data that is gold, when you need to externalise it? I'm getting requests right now from banks to do this events stuff, but we're not ready yet. 

So on a common platform I would expect that I can share events across BT, across the board, and I'll say typically you're going to get greater value from the data there. Typically the business people get it, I don't have to persuade them as a stakeholder, if I say to them I'm going to get real time data into our data platform, they get it. I don't have to tell them, you're going to make X money by selling X phones because you can market better. 

But if we look at our products and services, I see events as crucial to all those things. For me, I know I'm biased, but I see this capability as a cornerstone to BT's continued survival and success.