Sponsored By
The influx of connected devices has created new opportunities but also many challenges for both service providers and enterprises in managing and analyzing big data. The variety and velocity of data streams is increasing rapidly and the volume of data has reached astronomical numbers -- network data now has to be measured in terabytes.
In an effort to reduce the complexity of data processing for service providers and enterprises, the Linux Foundation launched a collaborative project for an open source Platform for Network Data Analytics (PNDA) in August 2016. According to the announcement, the PNDA is designed around "next-generation, big data architecture patterns" and the goal of PNDA is to deliver an end-to-end solution that provides a simplified, scalable big data analytics platform that enables organizations to extract valuable insights from their data.
Marc Cohn, vice president of network strategy at The Linux Foundation spoke with Telco Transformation about PNDA's approach to simplifying the collection and analysis of big data.
Telco Transformation: Networks are taking in more data than ever before but still not extracting all the value that they can from it. Why is that the case and what is PNDA doing to help?
Marc Cohn: Growth has been enormous, and with 5G coming, it will grow even faster. That exponential rate of data growth has necessitated a rethinking of performance and management. It's no longer possible to rely on what has worked in the past because those solutions simply can't keep up with the huge volume, variety and velocity [of data] coming in on today's networks.
Tapping into the power of big data is a relatively new development for the networking and communications world. With PNDA, we are attempting to demystify the big data elements, raise awareness and make it easier to engage with other open source communities within the ecosystem.
TT: How does PNDA remove the complexity and difficulty of developing the data pipeline needed to gain analytical insight?
MC: The key problem we're trying to solve is exposing an entry point for big data in a community that is not an expert in that domain. We want to make it available to the networking world for those who understand the benefits of analytics, but not necessarily all the underlying components that make it work. We want to be able to abstract the details so that the applications developer who understands the concepts, but not the details of all the mechanisms, can apply the concept.
PNDA complements major open source software-defined networking [SDN], network functions virtualization [NFV] and network orchestration efforts such as OpenDaylight, Open Platform for NFV [OPNFV] and FD.io. Those who use it can take advantage of the tremendous potential inherent in big data without all the implementation-level understanding.
TT: While an applications developer may not need to become a big data expert to tap into the benefits of big data for the network, what are the key aspects about big data that they need to know?
MC: They need to know what the benefits are and what is normal for the network. If you know what's normal for a typical time and see something that deviates, you can have the system set up to respond automatically. Possibly, traffic can be redirected without suffering a real disruption. In contrast, in the past, when working in a static infrastructure environment a deviation would necessitate shutting everything down. In this new world, you can take advantage of the dynamism of the architecture and of the infrastructure to do things that are high value for those in operations and those on the business side.
It's one thing to monitor a switch port, but to be able to say that my business applications running on top of this are usually slow at certain times of day is the kind of discovery you can make as you move up the stack. Say you found out the networking device is completely congested and that it is from something occurring in Layer 2. With full information, you get a view of infrastructure capabilities, as well as capability and availability. That insight into the processes as they are happening is what makes it possible to dynamically reroute an alternative path that circumvents the problem area.
TT: How is scalability built into the design of PNDA?
MC: One of the ways is that we're using and leveraging existing big data technology that is already scaling for cloud performance. The various open source components that make up PNDA also create plug-ins to allow collection of data from other sources. Those have a lot of value because they eliminate the trouble to have to build up those connections.
TT: PNDA decouples data aggregations from data analysis and stores data in the rawest form possible. Why are those factors important for data analysis?
MC: We set it up to enable it to take the data and operate on that data as two separate functions because that setup allows for much greater flexibility than working off a set schema. In the past, one had to create very complex analytics databases with fragile schema that wouldn't hold up when a change was made. What we wanted to do is just allow an architect to find the data services that they're looking for, find the data storage and create the schema on the fly. If one tries to do everything a priori, then one remains constrained by the schema that was set, and that wouldn't allow for creating queries for analysis in near real time.
TT: What is the core value of the open source approach to PNDA?
MC: The open source approach is effective as a distribution and collaboration model. It allows for a common platform with value-added applications that can be proprietary and allow one to select from the best of breed options for different kinds of applications. That's a win-win for vendors and operators from a business standpoint. It's far more efficient than having each individual create proprietary platforms that don't integrate with others.
PNDA is freely available to anyone who wants to participate and we invite people to come to the site to look at the code and the content on the site. It's very easy to get involved, and there are no barriers to entry.
— Ariella Brown, Contributing Writer, Telco Transformation
