Will Swift Concurrency deploy back to older OSs?

If you read the Xcode 13 (beta 1) release notes, macOS 12 and iOS 15 is currently required when using async features, but that is listed under issues. So I assume Apple is attempting to resolve that issue (aka allow some form of backwards deployment of the async feature requirements).

Concurrency requires runtime support that does not backward deploy. The release notes imply that this is an "issue" that just needs to be fixed. It's not. It's a feature that would need to be implemented. At this time, folks should assume that concurrency does not backward deploy. That said, everyone is aware of the value of it doing so, and is something that is being explored/considered.

This is a language feature or an iOS version feature.

It is tempting to try to just put it into one bucket or another, but respectfully I feel that this is an oversimplification.

Concurrency, as a whole, is a language feature. It requires specific runtime support. Most languages, including Objective-C and C++, have language runtimes that provide logic that power many language features. On Apple platforms, that runtime support ships with the OS.

In the case of backward deployment on Apple platforms, it is about finding a way to provide the needed runtime components for concurrency when the underlying OS does not provide it. I'm not saying that is or isn't possible; Swift definitely has provided backward deployed runtime support for a variety of features before.

All I am saying is that is that backward deployment of concurrency earlier than iOS 15, as of right now, does not exist, and that the Xcode 13 release notes may accidentally be interpreted that this is a "known issue" rather than something that just isn't there. As folks need to make choices regarding their applications' minimum deployment targets, I just wanted to clarify that concurrency, as of right now, requires iOS 15 and higher.

I get there are reasons for this stance, but it seems like Apple is not really acknowledging the reality for developers in companies and mid sized teams that cannot act like small indies focused on one OS at a time or like internal teams in Apple maybe.

Seeing compose be developed in the open and back deploy back to Android 5 and Kotlin advance their concurrency efforts again backdeploying to very old Android versions and more and more Android OS components being updated on the Play Store and not requiring OS updates are reality for the industry I feel sometimes Apple is ignoring… same thing about Enteprise environments for devs, continuous delivery and automation expectations, etc… it would be nice if we could follow Apple’s train as they balze the new trail, but the reality is that in most mid to large software companies that are not mostly native app focused (very few) the native app developers are the thorny pariah / under represented and a bit resented minority that always does their own thing and does not really get with the program.

Sometimes Apple makes it very very hard not to say “oh well just use something like Flutter, Cordova/PhoneGap, or React Native” for the developers that would want to work on the native frameworks directly and give the best possible experience to the users.

If it requires special runtime support that will come out with iOS 15 and MacOS 12 then how was it possible to use those features with development toolchains? I was using async/await on Big Sur and it worked very well.

I am not sure about actors but supporting async await on older systems would come a long way and developers could replace their callback code.

If the features are not backwards compatible that means that a large portion of developers won't be able to use them for a couple of years as usually clients want to support at least 3 last OS versions.

I had no idea that Swift concurrency required iOS 15. It's on me to not have gotten that information in advance, but I suspect that it's going to be "news" for many Swift developers, and it's going to be terrible news because, in practice, the vast majority of iOS developers are simply not going to be able to use these features for years, and that I guess includes library authors, that won't be able to update libraries (that use asynchronous code) with async/await. Please explore/consider enabling developers to actually use these features in production code without waiting for years.

EDIT: I realized that, given that all iOS 13/14 compatible devices are still compatible with iOS 15 it's not really going to take years. It's OT, but ditching iOS 12 is going to be the big step for many developers, that could mean jumping directly to iOS 15 without leaving anybody out.

The lack of support for anything earlier than this years OS versions will definitely stop my company from investing any time & effort in Swift Concurrency for several years.
We support users on older OSs because a) its what our clients want, and b) its the right thing to do.

Ideally I'd support 2 years back (ie move to iOS 13 minimum this year) but in reality we go back as far as iOS 10 and get push back each time we drop a version.

That means the plans I had for a major refactoring with Swift 5.5 concurrency will be dropped - there's no way we want a code base with #if available all over it - the idea of the concurrency changes was to make the code cleaner.

Internal macOS tools might get to use it, but probably not encouraged as it will disconnect from the core iOS work and change patterns of development.

See you in ~2025...

That said, everyone is aware of the value of it doing so, and is something that is being explored/considered.

I'm glad to hear this is being explored! If there is any way to make this happen, the community would be incredibly appreciative.

Most places I've worked support [current iOS major version] minus 2 or 3. That's a long time to have to wait to touch any of this, especially given the enormous benefits. Yes, we could #if available, add complementary async APIs, etc. But the reality is we don't have the bandwidth to support so much duplicate surface area.

It seems disingenuous to argue that only the new OS’s provide the foundation for this language feature, as has been pointed out above, the tool chain currently allows building and testing on Big Sur.

Please don't describe such rationales as disingenuous. There are legitimate technical reasons why experimental toolchains can work but a general solution would differ, without calling into question the good faith of the person giving the explanation.

The technology for toolchains relies on overriding the system libraries on macOS and simulators, but this technology does not work in the general context of all apps deployed to all devices. A different technology would be needed to shim in just the specific capabilities needed to support concurrency backward-deploying. It may be possible, but it is certainly far more complex than the toolchain solution, and there would be many edge cases to be thought through and coded around.

Can you give any more information about which parts of the concurrency design might be particularly difficult to back-deploy?

Say, if features such as async/await and actors were limited to an application’s own code (and its bundled dependencies) and had no direct support from the SDK. Granted, the experience would be far from optimal, and I appreciate that the team is exploring ideas which may be much better than that, but perhaps a commitment to at least that basic level of support would allow people to invest more time using swift concurrency.

Alternatively, is there any timeline for when we should expect further details? Before iOS15 ships?

As was noted in the previous thread, concurrency is the biggest feature to go through evolution, and arguably the biggest change to happen to swift, period. It’s understandable that developers would like to start adopting it and proving feedback as soon as possible.

Please don't describe such rationales as disingenuous. There are legitimate technical reasons why experimental toolchains can work but a general solution would differ, without calling into question the good faith of the person giving the explanation.

Apologies, no offense intended. Thanks for the clarification on the difference between toolchain support and system level support.

I really hope system level support can be provided at least for the current generation os’s, if not even one prior. It will help the adoption of this technology enormously .

My old Macs have newer versions of the Java JDK and .Net runtimes than they do the Swift runtime. Isn't something seriously wrong here?

The cost of coupling the shipping something as fundamental as the language runtime with the OS version is totally brutal.

It is not right for me to be part of the force that accelerates the obsolescence of users' devices and turns them into e-waste, just because I wanted to use opaque return types, structured concurrency, or other such developer luxuries.

I feel like the Lorax, just think of the trees!

Well, one can choose which direction to expend the engineering force vector - full thrust ahead, or add a series of retardation maneuvers.

I think we should respect the effort involved in backporting new features to old platforms.

As a hypothetical example, I’d rather see a full fledged swift 6 in a years time with full thrust ahead rather than in two years if core contributors would spend time backporting the concurrency features to old os versions and to handle the inevitable fallout of bugs / corner cases.

Personally I prefer full thrust ahead as far as possible, to move the state-of-the-art forward faster.

Well, one can choose which direction to expend the engineering force vector - full thrust ahead, or add a series of retardation maneuvers.

Backwards compatibility isn’t just a “retardation manoeuvre”; it is critical to ensuring that Swift developers can actually use the feature, which in turn means that the core team can get feedback about how well it suits its intended purpose or whether anything needs adjusting (e.g. I expect actor reentrancy to be a big topic. How well will developers adjust? We’ll have to see).

Engineering isn’t a profession where you dream up a grand plan, entirely perfect at conception, and then just go away and build it. You constantly have to measure and adjust to realities that might only emerge as you build.

Fair enough, you are right in the sense that there is a tradeoff to be made, but it is enough that "enough" Swift developers can use it to provide feedback too, so exactly where to draw the line is always a fine art. I just wanted to point out that back porting features isn't for free and that you will inevitably lose momentum to some degree while doing it - and that we should respect the tradeoff that needs to be made.

I am well aware that engineering isn't a profession where you dream up a grand plan which is perfect at conception and build it, thanks - the important thing is just to get enough valid feedback and draw the line somewhere. I don't think anyone would argue for back porting everything 5 releases back. Especially since there are statistics 5 - if 85%+ of all devices already run the latest iOS release (14), it seems likely that the same adoption rate will happen for the next release.

(I too believe that the actor reentrancy will be very interesting to follow - it seems likely to cause some trouble for many FWIW)

Not having all the details on the effort involved makes it impossible to second-guess such choices, but given that there are still a lot of fundamentally big things still missing (like e.g. memory ownership), I hope the pendulum will be swinging more towards forward thrust at this point - definitely understand others may have different views of course.

That said, everyone is aware of the value of it doing so, and is something that is being explored/considered.

First of all, I want to thank the team for such a great concurrency model!

It is also good to hear backwards compatibility is being explored/considered, and I just wanted to add my +1 request for it.

While undesirable, I do think it is OK for us to work around & deal with the lack of backwards compatibility of Apple frameworks. The community/devs can build wrappers, reimplement some things and provide fallbacks for most of those things. However, if the language becomes the problem, there is nothing we can do but wait several years, which is extremely sad, given how cool, useful & convenient the Swift concurrency is

Well, AFAICT it would require a new runtime, and the implication of shipping Swift runtime with the OSes has been noted before 36, albeit more generally:

Can I choose to bundle a newer Swift runtime with my apps going forward to be able to use new runtime features without requiring a new OS?

This will not be possible for a number of reasons: [...]

Is there anything that can be done to allow runtime support for new Swift features to be backward deployed to older OSes?

It may be possible for some kinds of runtime functionality to be backward deployed [...] The Core Team will consider the backward deployment implications of new proposals under review on a case-by-case basis going forward.

but they're not ruling this out as impossible, if that's what you want.

If you compile a project a particular toolchain, you gotta make sure there's a supported runtime installed on the target machine as well.

Edit:
But I digress, this thread isn't really about ABI stability. And they probably already received feedbacks that some places would like back-deploy.

It doesn't have to be a shim, either; at the very least it could just be backported as an entire update to older OSes, i.e. iOS 14.6.1 (if 14.6 were otherwise the last version of iOS before 15) and iOS 13.7.1, right? This is done all the time for security fixes, why not OS features?

That's not going to work IMO as it requires the users to have updated, and the gating of the #available goes well beyond the version number and would have to be changed to be feature based.

The updates need to be shipped as a library/bodge with the app that 'polyfills' the features onto older runtimes. ISTR that early Swift did this with the whole runtime to get working on devices before it stabilised and got bound to the OS - seems like we need to go backwards a bit to move forwards.

No, it isn't one-to-one with completion handlers. The key difference is that functions that take a completion handler typically return immediately and the completion handler is triggered later (though you can't assume that the completion handler isn't called before the function returns; in some cases, for instance if results are already available, for instance from a cache, it might be).

With an async function, it won't return until execution gets to a return statement, falls off the end of the function, or the function throws. Instead, when an async function does an await, the thread it's on is freed up to do other work (i.e. other async functions may resume execution, and on the main thread/actor, the application's main run loop will run so that the UI remains responsive).