attempt at implementing #10

[kianenigma]

This is my attempt at implementing #hidden. It is not working, and I think it is because my approach might fundamentally be incompatible with the intended way.

In short, what I did was to start parsing syn::File and remove any item/expr that comes after #[docify::hidden]. This part is intentioanlly left incomplete (it does not parse nested expressions like syn::Block).

Nonetheless, this is not working because source_excerpt (for reasons beyond my current understanding) was expecting to be fed the untouched source_code, not the modified version.

For a similar reason, all of my formatting was also being wiped. I like the current approach of always formatting the coment and I think it should be the default behavior. We can backport this part to master sooner.

[sam0x17]

ah yeah this would have to integrate closely with source_excerpt to work properly. At the top of my head I believe you would need to do your removals in a clean way that doesn't destroy formatting and then feed that into the regular docify parsing process. The CompressedString machinery has the tools you need to do this, as it lets you identify the "real" start and end position of an item within the original source string. If you take a look at the source code for source_excerpt it basically does exactly that using CompressedString

[sam0x17]

would be much easier to use a visitor pattern for this, see https://docs.rs/syn/latest/syn/visit/index.html and other visitors in this file

[kianenigma]

I had it in the back of my head that doing this nested parsing probably has a proper algorithm somewhere and I should not need to re-invent it 😅 So it is the same visitor alg you have been talking about lately, thanks for sharing

[sam0x17]

In fact, it was my primary area of research in grad school and I've published some novel indexing data structures and algorithms around this problem that let you do it without having to have an awkward visitor pattern :)

• https://dickinson.hykucommons.org/concern/etds/97a2026c-684f-4fbc-82eb-6241931a1897?q=AST%20indexing
• https://cs.brown.edu/research/pubs/theses/masters/2017/kelly.samuel.pdf

[sam0x17]

btw, this is one of the things sin would make easier. Sooo many methods that should be provided by default but aren't