7

Amazon S3 現在變成 Strong Read-After-Write Consistency 啦… – Gea-Suan Lin's BLOG

 3 years ago
source link: https://blog.gslin.org/archives/2020/12/02/9852/amazon-s3-%E7%8F%BE%E5%9C%A8%E8%AE%8A%E6%88%90-strong-read-after-write-consistency-%E5%95%A6/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Amazon S3 現在變成 Strong Read-After-Write Consistency 啦...

看到 Amazon S3 宣佈 Strong Read-After-Write Consistency 了:「Amazon S3 Update – Strong Read-After-Write Consistency」。

這個問題從很久前就被討論過:

所以到這次更新之前,只有新增的 object 會保證馬上出現。現在則是 update 也會:

Effective immediately, all S3 GET, PUT, and LIST operations, as well as operations that change object tags, ACLs, or metadata, are now strongly consistent. What you write is what you will read, and the results of a LIST will be an accurate reflection of what’s in the bucket. This applies to all existing and new S3 objects, works in all regions, and is available to you at no extra charge! There’s no impact on performance, you can update an object hundreds of times per second if you’d like, and there are no global dependencies.

要注意這邊沒有提到 DELETE,所以有可能 DELETE + GET 的操作還是沒有到 strong consistency,不過句子本身意思不是很清晰,也許這幾天會有人在 forum 上面問然後有答案...

另外從公告裡面提到 Amazon EMR 團隊,看起來是 Amazon EMR 團隊一直在內部戳 Amazon S3 的團隊改善:

We’ve been working with the Amazon EMR team and developers in the open-source community to ensure that customers can take advantage of this update with their big data workloads. As a result of that you no longer need to use EMRFS Consistent View or S3Guard, further reducing the cost to run big data workloads in AWS.

Related

Amazon S3 變成 Strong Consistency 背後的改善方式

看到 Hacker News 上的討論「Diving Deep on S3 Consistency (allthingsdistributed.com)」才想到該整理一下,原文的「Diving Deep on S3 Consistency」是 Amazon 的 CTO Werner Vogels 花了一些篇幅描述 Amazon S3 怎麼把 Eventually Consistent 變成 Strongly Consistent,當初 Amazon S3 公告時我也有寫一篇文章提到:「Amazon S3 現在變成 Strong Read-After-Write Consistency 啦...」。 Amazon S3 之所以會是 Eventually Consisient 是因為 Metadata Subsystem 的 cache 設計: Per-object metadata is…

April 28, 2021

In "AWS"

Amazon S3 的改善

在「Amazon S3 Introduces New Usability Enhancements」這邊提到了 Amazon S3 的兩個改善。 第一個是業務面的改善,以前應該是開 support ticket 請人調整 S3 bucket 數量上限,現在則是可以直接透過界面申請?(沒有遇過瓶頸,不知道以前是不是在界面上看不到...) 第二個才是重頭戲:Read-after-write Consistency。 With this enhancement, Amazon S3 now supports read-after-write consistency in all regions for new objects added to Amazon S3. 也就是說,Amazon S3 現在保證「新增的 object」「可以在建立後馬上被讀取」。之前還沒修正前,這個問題有多嚴重呢?可以看 2014 年時「Netflix 對 S3 的 Eventually Consistency 的補強...」這邊 Netflix…

August 5, 2015

In "AWS"

Netflix 對 S3 的 Eventually Consistency 的補強...

眾所皆知的,Netflix 幾乎將所有服務都放在 AWS 上,這當然也包括了 Amazon S3。 在 Amazon S3 上會有 Eventually Consistency 的問題:寫入後可能會讀到舊的資料,於是就算錯資料了... Netflix 的人討論了幾種方案,後來開發 s3mper 用來解決 Amazon S3 的 Eventually Consistency 問題:「S3mper: Consistency in the Cloud」。 s3mper 透過 AWS DynamoDB 儲存檔案的 metadata,藉以得知是否 consistency。而 Amazon DynamoDB 本身雖然也是 Eventually Consistency,但多了 API 可以得知是否 Consistency。 在 Supported Operations in DynamoDB 可以看到 Data Read…

January 12, 2014

In "AWS"

a611ee8db44c8d03a20edf0bf5a71d80?s=49&d=identicon&r=gAuthor Gea-Suan LinPosted on December 2, 2020Categories Cloud, Computer, Murmuring, Network, ServiceTags after, amazon, aws, cloud, consistency, read, s3, service, strong, write

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Comment

Name *

Email *

Website

Notify me of follow-up comments by email.

Notify me of new posts by email.

Post navigation


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK