47

meta: WebAssembly ("wasm") support · Issue #18892 · golang/go · GitHub

 6 years ago
source link: https://github.com/golang/go/issues/18892#issuecomment-359208503
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

WebAssembly ("wasm") support · Issue #18892 · golang/go · GitHub

all: WebAssembly ("wasm") support #18892

Closed

bradfitz opened this issue Feb 2, 2017 · 147 comments

Closed

all: WebAssembly ("wasm") support #18892

bradfitz opened this issue Feb 2, 2017 · 147 comments

Comments

Contributor

WebAssembly ("wasm") is similar to Native Client, but different notably in that other browsers plan to implement it.

http://webassembly.org/

This has been asked about a few times, so this is a tracking bug for it.

Whether we get it via cmd/compile, gccgo, or llvm-go, we can post updates here.

dsnet, mvdan, bacher, heapwolf, renannprado, ohenepee, abourget, themihai, bketelsen, mamiu, and 630 more reacted with thumbs up emojidemizer, icholy, RaananHadar, jacktuck, jershell, brianvoe, brettlangdon, tarndt, ehmo, teh-cmc, and 136 more reacted with hooray emoji1l0, ernado, bacher, heapwolf, renannprado, mamiu, dmonay, alepacheco, dmitshur, tiborvass, and 211 more reacted with heart emoji

Member

minux

commented

Feb 2, 2017

via email

@cherrymui and I discussed the possibility of a wasm GC port at length in the past few days:

Our current conclusion is that there is no efficient way to implement setjmp/longjmp functionality currently, therefore it's not a viable target of a gc port. We need to wait until it gets real stack unwinding and exception handling support.

All the other aspects looked fine, and we even designed a Go ABI under wasm (pass g and SP on wasm stack and everything else on emulated Go stack) and a way to modify the current MIPS backend to support wasm by emulating MIPS registers as wasm local variables.

Perhaps GopherJS can support wasm easier.

1l0, tomheng, audathuynh, tiborvass, icholy, irake99, tmthrgd, KirinHuang, devsmt, jaracil, and 5 more reacted with thumbs up emojilinkerlin reacted with laugh emoji

Contributor

From gopherjs/gopherjs#432:

However, it is not a technology that GopherJS can use. GopherJS compiles on the AST level, not on the machine code level.

WebAssembly could be used as a backend for the normal Go compiler

yanpozka, dmitshur, SCKelemen, espectro, icholy, linkerlin, and qm3ster reacted with thumbs up emojiRangerMauve, eloff, deepblue-phoenix, dmitshur, SCKelemen, ip, egorse, ARwMq9b6, crvv, anjmao, and 7 more reacted with laugh emoji

Member

@minux The lack of threads or async i/o seems to be a bigger problem than the ABI workarounds we'd have to do because of wasm quirks.

Member

@minux, @cherrymui
could you post somewhere in more details what you did and achieved?
as you may know the go-interpreter community is considering writing an interpreter based on either LLVM or wasm bytecode.
(we may even have a GSoC student working on the wasm bytecode interpreter [1], so, just the "bytecode consumption" part, not the "bytecode production via some Go-based toolchain" one)

minimizing the amount of duplicated efforts and impedance mismatch between various components would be great.

dmitshur, gouthamve, mugli, YamiOdymel, dennwc, mccoder, alihammad-gist, johanbrandhorst, zet4, hooluupog, and 12 more reacted with thumbs up emoji

I highly recommend this as a strategic feature. Consider that in just a few years web programmers will flock in hordes to pick their language of choice to be compiled into web assembly. The sooner we join the race the better. Mozilla is pushing Rust hard as the web assembly language of choice...

ernado, mccoder, levrik, ross-spencer, abourget, dmonay, carwyn, leeola, hooluupog, tj, and 121 more reacted with thumbs up emojidlsniper, RaananHadar, and SCKelemen reacted with thumbs down emoji

I second what @RaananHadar said. It would be a pity if the spec didn't suit Go's needs, especially given how Go's runtime can be unique.

EDIT: Apologies for the MeToo nature of this comment :) Thanks for the pointer Brad.

dlsniper, VoyTechnology, ip, jacobtomlinson, and yask123 reacted with thumbs down emoji

Contributor

Author

Reminder: https://golang.org/wiki/NoMeToo

yields, mvdan, UnnoTed, nicerobot, RaananHadar, leafi, linniksa, benkaiser, dvrkps, microo8, and 21 more reacted with thumbs up emojidemizer, SCKelemen, pavelnikolov, aleksarias, syed, icholy, mateusdemboski, and iamveritas reacted with heart emoji

The WebAssembly stack machine context consists of linear memory and an operation stack. IIUC, saving the VM context can possibly be done by saving the:

  1. Current PC, bytecode string
  2. Linear memory, and the
  3. Operation Stack

To a global list of saved contexts in the VM context struct, if any.

Thus, a setjump should simply restore these values, and continue with the interpreter loop as usual (This is similar to how the emacs bytecode VM implements signals: https://github.com/emacs-mirror/emacs/blob/master/src/bytecode.c#L785). An exception system can be implemented on the top of this, though I'm not sure how performant would this be.

Further, the webassembly spec mentions that the stack height at any moment in the bytecode is statically known, making all stack operations equivalent to operations on registers unique to each position on the stack. This paper describes a way to achieve such a stack -> register mapping for converting stack machine code to native code, allowing us to use setjmp/longjmp as usual.

Edit: What about using panic() and recover() with special WasmException{} values for signalling an exception? recover does the tough job of stack unwinding already, it shouldn't be hard to restore the interpreter loop after recovering from a caught exception.

sprt, SoftwareDreamer, vpereira, vine77, sbinet, yanpozka, pannous, and radare reacted with thumbs up emojibradfitz reacted with heart emoji

Member

minux

commented

Mar 13, 2017

via email

Yes, basically the discussion with @cherrymui resulted in similar findings.

The initial design with @cherrymui is this: (this stage focus on getting $GOROOT/test/sieve.go to work, no async IO is considered.) we reuse the MIPS port, mapping the 31 MIPS registers as WASM local variables and only use WASM stack for intermediate results. We can't use WASM's function call mechanism for most Go functions because the lack of stack unwinding support. change the mips objlink (cmd/internal/obj) to emulate each MIPS instruction with respective WASM instructions. This makes setjmp/longjmp (aka runtime.gogo) implementation trivial and reuses a lot of existing effort.

Go already needs to implement its own stack, separate of the WASM stack, because Go needs copyable stack, stack pointer maps and stack unwinding support, which are not available from WASM. (This strategy also aligns with the LLVM port, we can't use LLVM stack for almost the same reasons (unless we add a custom backend pass to LLVM proper to encode stack pointer maps).)

We will probably need to use a big switch to thread indirect jumps? This is similar to NestedVM's approach. http://nestedvm.ibex.org/

If we decide to implement a native WASM backend, then we can do this: the Go ABI is: all parameters (including results) on (emulated) Go stack, at function entry, the following are on the WASM stack: emulated SP, g.

The discussion happened more than one month ago, and I probably forgot some details at this point.

dmitshur, vibhavp, kanocz, yookoala, yanpozka, SCKelemen, gaigepr, KonstantinCodes, and linkerlin reacted with thumbs up emoji

IIUC, you can completely discard the WASM stack (according to the spec). An extendable and copyable go stack is still something that needs to be investigated.

Member

minux

commented

Mar 13, 2017

via email

Yes, the initial design I mentioned only uses the WASM stack for intermediates used in computation. All data is either on Go heap (linear memory) or pseudo registers in WASM local variables.

I don't expect WASM spec to incorporate moving/segmented and copying stack. It's rarely used by mainstream languages targeted by WASM.

@bradfitz @minux @vibhavp What is the current state of wasm in golang? Nothing has happened since March 13th ! will it be possible transform golang to wasm in the future? is there maybe any route-map of that?

CAFxX and linkerlin reacted with thumbs down emoji

if somebody will decide to implement wasm support I can offer my time for non-global tasks

Contributor

Author

@SerkanSipahi, nobody is working on this. This bug is marked LongTerm. If something starts happening, you'll see updates here.

Contributor

I believe that NaCl is used by various parts of the Go ecosystem. That is, it's not used only by Chrome. And in any case that blog announcement is about PNaCl, which, although it provides functionality similar to NaCl, is actually a completely different product based on a different technology. So removing NaCl support from Go is premature at present.

Any in any case whether or not we remove NaCl from Go has nothing to do with whether we add support for Web Assembly.

I would say that PNaCl is just an extension over NaCl to provide platform independent code https://developer.chrome.com/native-client/nacl-and-pnacl I asked about NaCl deprecation here - https://groups.google.com/d/topic/native-client-discuss/wgN2ketXybQ/discussion

If is possible to get list of those various parts of the Go ecosystem where NaCl is still used?
Going through it one by one will make transitioning to WebAssembly more interesting.

Member

4ad

commented

Jun 7, 2017

edited

I would say that PNaCl is just an extension over NaCl

You can say whatever you want, but it doesn't mean you are right. They are completely different ISAs, NaCL is just the target ISA (x86/arm) with some restrictions, while PNaCl is a completely different ISA for an abstract machine.

Go has never supported PNaCL, and the Go NaCL port was never suitable for Chrome anyway. It had nothing to do with Chrome, it was not usable in the browser. Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance. How many more times this must be said? Of course, if NaCL is abandoned completely then Go will be forced at some point to abandon it too.

WebAssembly support in Go has absolutely nothing to do with NaCL. Go might or might not get WebAssembly support. If, or when it might get such support has no relevance to the state of the NaCL port.

dlsniper, mvdan, tmthrgd, and xrhstosgr reacted with thumbs up emojiSCKelemen reacted with confused emojidlsniper reacted with heart emoji

Contributor

Author

Here is a very important part of the ecosystem where we use nacl: https://play.golang.org/p/MfJIq8wb5-

(that runs server-side, not nacl-in-a-browser)

tmthrgd, yanpozka, twitchyliquid64, pradeep-raja, fgm, and icholy reacted with thumbs up emoji

Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance.

And who will support sandbox loader and toolchain then? It is imported from Chrome project, which switched efforts to WebAssembly. Wouldn't it be wise to follow and see if Go playground can already be ported from NaCl sandbox to WebAssembly sandbox? It can also be run server-side.

Contributor

I think everybody is in favor of supporting WebAssembly.

The time to discuss moving away from NaCl is after WebAssembly is fully working. There's no point to discussing it before then.

vine77, techtonik, leeola, matejb, tmthrgd, paulm17, inkeliz, wheelcomplex, davidlanouette, aleksarias, and 2 more reacted with thumbs up emoji

@ianlancetaylor, can you clarify "after WebAssembly is fully working"?

At a glance, it appears webassembly.org says "the initial version of WebAssembly has reached cross-browser consensus" and wasm is available in current or future browser versions from all browser vendors.

techtonik, ernado, and foresmac reacted with thumbs up emojitv42 and icholy reacted with thumbs down emoji

Contributor

@vine77 I mean after WebAssembly is fully working for Go programs, at least as well as NaCl works today.

Member

I hope everyone had nice holidays. Here's the promised update:

Since my last update, I managed to get reflection, stack-growing and the garbage collection up and running. Additional improvements got it to the point that the tests of 104 out of 136 stdlib packages are passing now. Also, many of the compiler tests are green. Good progress!

Please keep in mind that I am currently focusing on feature support and correctness. I haven't done any optimizations for performance or output size yet.

kanocz, 1l0, j7b, Majkl578, lukeed, cretz, 13rac1, PierBover, jhsx, nightlyone, and 160 more reacted with thumbs up emojiSerkanSipahi, brunocassol, johanbrandhorst, RobbieMcKinstry, dvrkps, Equim-chan, ACollectionOfAtoms, akavel, fortytw2, zebaz, and 12 more reacted with laugh emojikanocz, 1l0, tj, theSoenke, nightlyone, wader, mini-eggs, tmthrgd, KenjiTakahashi, dmitshur, and 73 more reacted with hooray emoji13rac1, msample, tmthrgd, dmitshur, TUSF, agnivade, marwan-at-work, vith, enj, dmonay, and 59 more reacted with heart emoji

Contributor

@neelance - Now that callbacks are in, I believe basic support is complete. Shall we close this bug or did you have something else in mind ?

Member

Yes, I think we can close this now. tada

mattn, ruedap, Hecatoncheir, johanbrandhorst, Jinex2012, tani, ernado, QingLeiLi, Merovius, optim-nakano, and 67 more reacted with thumbs up emojimattn, siygle, ruedap, keegancsmith, nlepage, tsuzu, johanbrandhorst, Jinex2012, 1l0, ernado, and 71 more reacted with hooray emojimattn, ruedap, johanbrandhorst, Jinex2012, ernado, QingLeiLi, acid-chicken, danihodovic, jansowinski, shahinv, and 34 more reacted with heart emoji

Great work!!! @neelance

QingLeiLi, danihodovic, dmonay, andybons, IncSW, and SCKelemen reacted with thumbs up emoji

This is awesome, great work @neelance!

dmonay, jbub, vladimirvivien, IncSW, SCKelemen, postspectacular, anitgandhi, and nightlyone reacted with thumbs up emoji

Change https://golang.org/cl/120057 mentions this issue: doc/go1.11: mention GOOS/GOARCH values of WebAssembly port explicitly

Change https://golang.org/cl/120575 mentions this issue: cmd/dist: skip non-std tests on js/wasm

Is it appropriate to open an issue on reducing the file size (if that is even possible)? Or is that being tracked elsewhere?

Contributor

Author

@matthewp, feel free to open a webassembly-specific size bug if you'd like. The general one is #6853.

matthewp reacted with thumbs up emoji

Change https://golang.org/cl/120958 mentions this issue: net: re-implement built-in simulated network on JS and NaCl

Does the golang compiler have built-in control flow integrity(CFI)? I recently skimmed through the following to do with vulnerabilities in WASM prevented with CFI built-into the clang compiler:
https://www.fastly.com/blog/hijacking-control-flow-webassembly-program
https://github.com/trailofbits/clang-cfi-showcase/blob/master/cfi_vcall.cpp

Member

Just skimmed it, but it seems to me as if the post says "WebAssembly is not a high level language such as JavaScript, but a low level language, so certain classes of bugs may apply if the source language (C or Go) allows them." No surprise there. This means that if Go would have such issues, they would also apply to other architectures, not only WebAssembly.

Contributor

WebAssembly is mostly about defending the browser from nefarious WebAssembly code. It is not so much about providing guarantees within the WebAsssembly instance (although there is some of that). As @neelance said, it is up to the LanguageX->WebAssembly compilers to provide any required safety guarantees.
Go does not have CFI. I don't think it would be easy - at the point where a method starts, we've already lost the vtable. We've even lost the fact that a vtable was used.
Go is vulnerable to this exploit, in cases where there is any parallelism. For now, at least, the WebAssembly Go port (and WebAssembly, full stop) does not have parallelism so this exploit is only theoretical. The WebAssembly folks do plan to implement parallelism at some point, though.

brtn reacted with thumbs up emoji

Member

@randall77 How does this affect plain linux/amd64? Is the risk somehow larger for WebAssembly?

Contributor

The risk is the same regardless of architecture (with the exception that WebAssembly doesn't have threads yet, so it's actually zero for WebAssembly for the moment). Data races can be used to simulate unsafe, without importing unsafe.
It would be hard to exploit such a vulnerability. You either need to link in untrusted code, or somehow find and trigger a gadget in an existing binary that does a data race on an interface value and then invoke a method on the result.

Member

Okay, so if your Go code has no data races, then it should be fine. Also, when running untrusted code with WebAssembly you can make sure that it can not escape the WebAssembly environment, even though it is able to have data races or simply use unsafe. Thanks for the interesting insights and sorry for this being slightly off-topic for this issue.

The more I examine this javascript usage from within golang, the more I perceive as an infection that will degrade the quality and the long-term maintenance of the golang code. Let me explain. Here is an example of a wonderful wasm golang app that does some impressive eye candy magic.

https://github.com/stdiopt/gowasm-experiments/blob/master/bouncy/main.go

The results are impressive. Delving furthing into the code that actually makes it happen though was disappointing because it require that every time you want to call something from the javascript side, it requires you to describe the name of the javascript function or the javascript value as a string AT EVERY TURN. As a result, there is no way for the golang compiler to check javascript correctness at compile time. It's "fly by prayer" at runtime. I would prefer to see golang functions/variables/types everywhere for long-term maintenance otherwise it defeats the purpose of using golang for wasm in my humble opinion. BTW I detest javascript and always have. It's too freestyle. Why is it that such a language has persisted in the web browsers? The answer escapes me.

Thank you for listening.

Merovius and zet4 reacted with thumbs down emojipdf and zet4 reacted with confused emoji

I agree that calling JS using strings isn't the better experience. Maybe we could generate Go interfaces from web platform specification IDL's. The downside is how to keep up with the spec, because depending on the browser it will run, the API's isn't available, while on the other side, new spec keep coming all the time as the web platform evolves.

Member

@omac777 I agree with you. That's why I hope that there will be nice Go libraries around the JS low levels so most users will never have to touch syscall/js directly. Similar to the syscall package, it is ugly and almost no one uses it directly. ;-)

zet4 reacted with thumbs up emojizet4 reacted with heart emoji

Contributor

@omac777 For GopherJS, many people use high-level bindings for various browser APIs rather than the low-level js package directly. I suspect a similar approach will be most popular with Wasm too. I expect many of the GopherJS bindings will add support for Wasm in the same package, and new packages will be created as needed.

For reference, see https://github.com/gopherjs/gopherjs/wiki/Bindings, https://dmitri.shuralyov.com/talks/2016/Go-in-the-browser/Go-in-the-browser.slide#11 (and the following 4 slides), and dominikh/go-js-dom#57.

geovanisouza92 reacted with heart emoji

Also we should consider there is a bunch of stuff on the webassembly roadmap:

https://webassembly.org/docs/future-features/

that will improve the syscall story quite a lot, in the long run, like

reference DOM and other Web API objects directly from WebAssembly code;
call Web APIs (passing primitives or DOM/GC/Web API objects) directly from WebAssembly without calling through JavaScript; and
efficiently allocate and manipulate GC objects directly from WebAssembly code.

Anyways, thanks @neelance et. al. for implementing the initial version. So awesome! 1st_place_medal

What if there was a Go implementation of the DOM?

Documents and scripts could be written and executed in Go, making use of the type system.

Then, code could be generated from the Go DOM.

@fractalbach: If the WebAssembly's host binding is confirmed and implemented by the work group, you can actually do DOM manipulation through WebAssembly. Then it would be reasonable to have high level libraries to abstract DOM, Documents and scripts.

But before that, it is not quite as appealing to do.

Go WebAssembly: Binding structures to JS references

https://medium.com/@nlepage/go-webassembly-binding-structures-to-js-references-4eddd6fd4d23

The approach is appealing. The usage is similar to json marshallling/unmarshalling within structs. That makes it an easily transferable skill.

The one thing missing is how to apply the same approach to duck-typing? I would prefer not to have the function declarations within the structure, but outside of it standalone in the same manner as duck-typing. If the service exists, it can use it. That way it is easily modifiable without affecting the core struct holding the different desired non-function values. Duck-typing rocks!

Thank again @neelance for all your work. It's greatly appreciated.
Thank you Mr. Nicolas Lepage for your points of view. They really help to crystallize what could be a better path for interoperating with all this javascript noise for the interim until the direct wasm interfacing is complete.

Contributor

Hello everyone, this is a high traffic thread with a lot of people subscribed. Having a general discussion like this is not fruitful to those just concerned about the original issue - which is to have WebAssembly support for Go.

Please feel free to continue the discussion in appropriate forums - golang-nuts/golang-dev. Or if you have something specific in mind, please open a new issue.

Thank you.

mvdan and tmthrgd reacted with thumbs up emoji

golang

locked as resolved and limited conversation to collaborators

Jul 6, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Assignees

No one assigned

Labels
arch-wasm WebAssembly issues NeedsFix The path to resolution is known, but the work has not been done.
Projects

None yet

Milestone

Go1.11

Development

No branches or pull requests

49 participants
and others

Footer

Member neelance commented

I hope everyone had nice holidays. Here's the promised update:

Since my last update, I managed to get reflection, stack-growing and the garbage collection up and running. Additional improvements got it to the point that the tests of 104 out of 136 stdlib packages are passing now. Also, many of the compiler tests are green. Good progress!

Please keep in mind that I am currently focusing on feature support and correctness. I haven't done any optimizations for performance or output size yet.

170 kanocz, 1l0, j7b, Majkl578, lukeed, cretz, 13rac1, PierBover, jhsx, nightlyone, and 160 more reacted with thumbs up emoji 22 SerkanSipahi, brunocassol, johanbrandhorst, RobbieMcKinstry, dvrkps, Equim-chan, ACollectionOfAtoms, akavel, fortytw2, zebaz, and 12 more reacted with laugh emoji 83 kanocz, 1l0, tj, theSoenke, nightlyone, wader, mini-eggs, tmthrgd, KenjiTakahashi, dmitshur, and 73 more reacted with hooray emoji 69 13rac1, msample, tmthrgd, dmitshur, TUSF, agnivade, marwan-at-work, vith, enj, dmonay, and 59 more reacted with heart emoji


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK