Github Scrape code examples from `examples/` directory for Rustdoc by willcricht...

jyn514 commented 3 days ago

@shepmaster I don't think those issues are closely related. Those are about making it easier to manually write examples and guides, this is about automatically generating docs for existing examples. Manual docs will always be used less than automatic docs just because they take more effort to write.

shepmaster commented 3 days ago

What prevents MyType::new from linking to every example? How do you choose which examples are in line and which are in the see more link?

willcrichton commented 3 days ago •

What prevents MyType::new from linking to every example?

Nothing -- under this proposal, MyType::new would be linked to every call to it. If every example calls MyType::new, then every example will get linked.

How do you choose which examples are in line and which are in the see more link?

Currently, it's arbitrary, whichever example happens to get found first. I'm open to suggestions for more systematic ways to decide how to sort the examples. We could try sorting the examples by the size of the source file?

GuillaumeGomez commented 2 days ago

In some crates, it could lead to a lot of links. I like idea though, not just a big fan of the current approach. If we wanted to use it on gtk-rs for example, some items would have a lot of examples, very likely too many.

Also, in your current implementation, you seem to link to the file using a github URL. I don't think this is the right approach considering that it would force us to support other repositories host websites (exposing us to URL scheme change if any), and that would simply not work if there is no website. I guess we could generate the examples in HTML like we do for source code?

willcrichton commented 2 days ago

In some crates, it could lead to a lot of links. I like idea though, not just a big fan of the current approach. If we wanted to use it on gtk-rs for example, some items would have a lot of examples, very likely too many.

How about a configurable limit on the number of examples for any given function? Like --max-scraped-examples N as a flag to Rustdoc.

Also, in your current implementation, you seem to link to the file using a github URL. I don't think this is the right approach considering that it would force us to support other repositories host websites (exposing us to URL scheme change if any), and that would simply not work if there is no website. I guess we could generate the examples in HTML like we do for source code?

Yes, I discussed this in the RFC. Currently I match on the domain of the package.repository field of the manifest and special-case generate a link to the source viewer for that repo. However, if Rustdoc were changed to include examples in HTML, then I could just link to those.

Nemo157 commented 2 days ago

Other related issues:

• rust-lang/docs.rs#132
• rust-lang/docs.rs#238 (the title says binary, but examples are just special cases of binaries and I mention in the issue that we could generate documentation for all the example crates in the package too).

For docs.rs specifically we already host all the source files from the package independently of rustdoc's source html output. So we could use something like --example-src-base-path=/crate/foobar/0.1.0/source/ to tell rustdoc how to generate links to the source files.

GuillaumeGomez commented 2 days ago

How about a configurable limit on the number of examples for any given function? Like --max-scraped-examples N as a flag to Rustdoc.

That's an option on top of another option, definitely not a fan. I think the solution here would be in the UI.

Yes, I discussed this in the RFC. Currently I match on the domain of the package.repository field of the manifest and special-case generate a link to the source viewer for that repo. However, if Rustdoc were changed to include examples in HTML, then I could just link to those.

Well, nothing prevents us to generate it. Currently we don't because we have no use for them.

shepmaster commented 2 days ago

I hope to leave a more useful comment at some point, but I wanted to at least jot these thoughts down before I lose them:

• It would be nice to extend this to all code blocks in the docs.
• It would be nice to click on the types / methods inside the code block and link back to the function.
• I worry about the medium- and long-tail of customization. Some possible things people are going to want:
  • #[cfg] directives
  • the ability to hide / show pieces of code
  • the ability to add prose into the example (rustdoc /// comments inside the example; recursively. shudder)

@shepmaster Seeing how difficult it is to get rust-lang/rust#84176 merged because some rustdoc team members aren't very favorable to the feature, I think it's going to be complicated (even though I agree).

jyn514 commented 17 minutes ago

I hope to leave a more useful comment at some point, but I wanted to at least jot these thoughts down before I lose them:

* It would be nice to extend this to _all_ code blocks in the docs.

I'm not sure what you mean by this - do you want to extend this to doctests, so that rustdoc links to other items in the same crate? That seems pretty hard ... Rustdoc doesn't analyze doctests, just pass them straight through to rustc, so it would need fundamental changes.

* It would be nice to click on the types / methods _inside_ the code block and link back to the function.

This seems like it could be part of rust-lang/rust#84176 (or added later during implementation). I don't think it needs to be part of the RFC directly, this is still useful without.

* I worry about the medium- and long-tail of customization. Some possible things people are going to want:
  
  * `#[cfg]` directives

I don't know what you mean by this.

  * the ability to hide / show pieces of code

Maybe this can be a per-item attribute? It's kind of tricky with re-exports though, see rust-lang/rust#84597 (comment).

  * the ability to add prose into the example (rustdoc `///` comments inside the example; recursively. _shudder_)

This seems like a misfeature, I don't know why we would support it.