InfoQ eMag: Modern Data Engineering
source link: https://www.infoq.com/minibooks/data-engineering/?itm_source=minibooks&itm_medium=link&itm_campaign=homepage
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Feb 22, 2021
About the Author
InfoQ.com is facilitating the spread of knowledge and innovation in professional software development. InfoQ content is currently published in English, Chinese, Japanese and Brazilian Portuguese. With a readership base of over 1,400,000 unique visitors per month reading content from 100 locally-based editors across the globe, we continue to build localized communities.
The InfoQ Newsletter
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example
Data architecture is being disrupted, echoing the evolution of software architecture over the past decade. The changes coming to data engineering will look and sound familiar to those who have watched monoliths be broken up into microservices: DevOps to DataOps; API Gateway to Data Gateway; Service Mesh to Data Mesh. While this will have benefits in agility and productivity, it will come with a cost of understanding and supporting a next-generation data architecture.
Data engineers and software architects will benefit from the guidance of the experts in this eMag as they discuss various aspects of breaking down traditional silos that defined where data lived, how data systems were built and managed, and how data flows in and out of the system.
We would love to receive your feedback via [email protected] or on Twitter about this eMag. I hope you have a great time reading it!
Free download
The InfoQ eMag - Modern Data Engineering include:
- The Future of Data Engineering- Chris Riccomini examines the current and future states of the art in data pipelines, data streaming, and data warehousing. He presents a six-stage evolution that data ecosystems follow from simple monolith to a complex data-microwarehouse architecture as the data engineers who manage them solve problems and clarify their roles as infrastructure engineers rather than data stewards.
- Beyond the Database, and beyond the Stream Processor: What's the Next Step for Data Management? - Databases have been around forever with the same shape: you make a request to your data and then you receive an answer. Now, stream processors came along with a different approach: data isn’t locked up, it is in motion. Understand how stream processors and databases relate and why there is an emerging new category of databases that focus on data that stays in place as well as data that moves.
- Data Gateways in the Cloud Native Era - Data Gateways act like API Gateways but focus on access to the data aspect. A Data Gateway offers abstractions, security, scaling, federation, and contract-driven development features. There are many types of Data Gateways, from the traditional data virtualization technologies, to light GraphQL translators, cloud-hosted services, connection pools, and fully open source alternatives.
- Combining DataOps and DevOps: Scale at Speed - DataOps is an extension of DevOps standards and processes into the data analytics world. It's about streamlining the processes involved in processing, analyzing and deriving value from big data.
InfoQ eMags are professionally designed, downloadable collections of popular InfoQ content - articles, interviews, presentations, and research - covering the latest software development technologies, trends, and topics.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK