大规模数据处理的演变(2003-2017)
source link: http://mp.weixin.qq.com/s/qVbqgssZllgJWCutMNE49A
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
大规模数据处理的演变(2003-2017)
介绍2003-2017年大规模数据处理的演变，通过时间的发展来表现整个大数据生态技术的发展过程，涉及到系统设计paper，包括未来发展的方向。重在带领大家探索时间驱动技术的演变过程，中间很多系统的权衡设计，分分合合的流处理+批处理，不同的框架对现有技术的改进和高质量的工业级的系统编码实现权衡。
最后说Apache Beam是统一Batch+Streaming的未来，重点介绍它的可移植性，能跑在Flink、Spark、Google Cloud DataFlow之上。
我们都允许不同的观点出现，Flink才是真正做到统一Batch+Streaming，他把Batch处理看做是一种特殊的Streaming，思想相对Spark Streaming已经是很大的进步了，从一开始就自己管理内存，性能不错，并且已经大规模开始应用。
观点和片子中指出Flink是解决”Open-source out-of-order”,有些不同，允许不同观点。 获取源PPT文件，请看文末。
PPT原文件：链接: https://pan.baidu.com/s/1midzKgC 密码: 9sbv
欢迎关注微信公众号，第一时间，阅读更多有关云计算、大数据文章。
原创文章，转载请注明： 转载自Itweet的博客本博客的文章集合: http://www.itweet.cn/blog/archive/
Recommend
-
26
你好，我是蔡元楠， 目前在 Google Brain 担任 AI Healthcare (人工智能的健康医疗应用) 领域资深工程师，也是极客时间
-
16
Look at any recent book on building enterprise applications (such as my recentP of EAA) and you'll find a breakdown of logic into multiple layers which separate out different parts of an enterprise application. Different...
-
5
FlashForward 2003 NYC Wrap-up Monday, July 14, 2003 Well, FlashForward 2003 NYC is over, and as usual it was a great conference. I had a blast hanging out with and getting to know everyone. It was particularly cool to g...
-
14
FlashForward NYC 2003 Keynote Thursday, July 10, 2003 Well, I have finally been able to catch my breath from the keynote (and from the lack of sleep over the past couple of days). Normally, I would give a blow by blow a...
-
14
FlashForward 2003 NYC Pictures Thursday, July 10, 2003 I have posted some pictures from FlashForward. They are mostly from the speaker reception, and the keynote. You can view them
-
15
Flash Forward NYC 2003 Tuesday, July 8, 2003 I just arrived in NYC for FlashForward 2003. I am really looking forwarding to hanging around with everyone, and have already run into Greg Burch and Branden Hall. I...
-
8
FlashForward 2003 San Francisco Wednesday, March 26, 2003 Well, today is the first day of FlashForward. I haven’t had a chance to get out to the conference yet (I am busy w...
-
10
Interaction 2003 Contest at ericd Tuesday, January 21, 2003 Just a quick heads up that you only have a couple of more days to submit content for the Interaction 2003 Flash contest...
-
2
本文由百度智能云大数据平台技术架构师——李莅在百度开发者沙龙线上分享的演讲内容整理而成。本次分享围绕云原生数据湖架构的价值展开，深度数据湖计算和统一元数据的技术架构。希望开发者能够通过本文对一站式大数据处理平台构建有初步认识。
-
8
大规模数据处理：探索如何高效地处理海量数据 作者：编程技术汇 2023-10-05 12:43:48 大数据 通过合理地选择和应用技术和方法，我们可以...
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK