Open Source: Turbocharged Digitization

From operating systems to security and data flow engines, transparency and inclusiveness are the essence of open source. By sharing global knowledge, developers are inspired to innovate, create and ultimately facilitate the optimization of organizational operations. Cloud computing echoes this spirit by providing flexible implementation and collaboration for open source projects, making them accessible to a wider audience.

While open source software has clear advantages, it’s not easy to get started without prior experience and support. In this context, the cloud makes it much easier to get started. Technology vendors can help developers begin their open source journey, such as by providing real-world experiments, free consulting, and integration and application lifecycle management support. All of this helps inject confidence right from the start of their journey.

At Alibaba Cloud, we embrace and pursue the open source philosophy holistically and consistently across platforms, industries and countries. We invest in both technology and knowledge development of the developer community in order to promote and accelerate digitization.

Driving stream computing in real time

Data processing and analytics has grown rapidly over the last decade and the trend is increasing. In the next two years alone, data volumes are expected to more than double. To handle the huge amount of information and make it applicable, powerful functions and calculation engines are needed to enable data processing in the shortest possible time.

With this in mind, Apache Flink, the open source technology for real-time stream computation, was developed. The software originated as a research project at the Technical University of Berlin, which was then donated to the Apache Software Foundation. In just a few years, Flink has become one of the fastest growing communities in the Apache Software Foundation. While traditional batch computing engines like Hadoop and Spark process finite datasets, Apache Flink focuses on stream computing and is capable of ingesting, analyzing, and processing data in real time to deliver impressive results as quickly as possible.

Using Flink as a framework, Alibaba has made several significant improvements in scalability, reliability, and performance, as well as developing its own Flink branch, called “Blink”. Soon after, Alibaba partnered with Ververica (formerly known as Data Artisans), the pioneer of real-time enterprise stream processing and analytics, and contributed “blinks” to the Apache Flink community. To date, Alibaba has contributed over a million lines of code to Flink and, together with Ververica, have contributed over 70% of the core improvements to the Flink community. Today, global companies like ING and Uber have used Flink as a flow processing engine to power stateful applications at scale, including real-time analytics, machine learning, content search and classification, and fraud detection.

For the benefit of AI development

As we look to the future, AI development in particular can further benefit from open source. In the open source community, global developers can participate in fine-tuning open source machine learning models, accelerating the pace at which tailored AI applications can be deployed to meet the diverse needs of enterprises across industries.

A more dynamic and creative ecosystem where developers can collaborate and exchange information is helpful for generating creative AI solutions. Small businesses or individual developers in particular can benefit from the open source community as they can access a wide range of tools and resources in a convenient way. This leads to further growth in the development and deployment of different AI applications and solutions.

Meanwhile, by making source code available, developers can also work together to more quickly identify and fix flaws within models, and jointly address common AI concerns around safety, transparency, security, and privacy. As a result, this can improve the reliability of AI tools and lead to more responsible development of AI models.

For example, Alibaba has made the source code of its latest federated learning platform FederatedScope, a federated learning platform with easy-to-use packages, accessible to the open source community. By sharing our self-developed federated learning technologies with the open source community, we hope to foster research and industrial implementation of privacy-preserving computing in different industries. These include the healthcare and smart mobility industries, which usually involve sensitive user data and require strict privacy protection practices.

Contribution to the developer community

Collaboration and contribution are critical to the sustained growth of the open source community. That’s why we’re actively involved in open source initiatives like the Cloud Native Computing Foundation (CNCF), the Linux Foundation, and RISC-V International. We’ve also contributed to more than 2,700 open source projects on Github.

With our newly launched Apsara global developer community, we have gone one step further to empower developers: on this platform, they can find resources and information that best equips them for digitization trends like AI or Web3. The community aims to bridge the gap between theory and practice, with training materials, tutorials, developer tools, and projects. Meanwhile, forums, blogs, community events, and hackathons provide opportunities for developers to network both online and offline and compete with like-minded people from all over the world. Therefore, we hope that even more developers globally can participate and contribute to the open source community.

Today, open source plays a vital role in driving the development of cloud computing, artificial intelligence and other frontier technologies. We believe the future of open source, and all that it can help deliver, is extremely promising. True to the principles of transparency and openness, the global open source community is set to grow at a faster and more turbo-charged pace as we all continue to contribute by sharing our advanced technologies, skills and capabilities.

#Open #Source #Turbocharged #Digitization

Leave a Comment