Our current -fdelayed-template-parsing and -fms-compatibility approach is
pretty hacky. We have lots of weird dependent nodes that don't normally
appear in an eagerly instantiated clang AST, and it adds a lot of complexity
without getting us all the way there in terms of compatibility.

For example, we have bugs like:
#18817
#16386
#13077

By reparsing the template with token instantiation and replay, we'd neatly
solve all of these problems, hopefully with less code and fewer bugs.

Hi Reid,

Can you give some pointers on the nature of the token transformation?

Do you have a feel for the phases of token caching, instantiation and substitution that might be going on in MSVC?

I'm not planning to work on this now, but if we have enough problems, this
is probably the right future direction for -fdelayed-template-parsing.

Can you give some pointers on the nature of the token transformation?

Do you have a feel for the phases of token caching, instantiation and
substitution that might be going on in MSVC?

First, I just want to say that I've been talked out of this approach since I filed this bug, so I'm closing it. :)

Answering your question, yes, they appear to do roughly what -fdelayed-template-parsing does: they parse class template bodies, but they slurp the tokens from C++ inline method definitions and parse them once they are instantiated and referenced. I believe you need to use typename with MSVC when appropriate in a class template body outside of C++ inline methods.

The key difference is that method bodies are never parsed prior to instantiation. The template parameters are substituted with arguments before parsing, so MSVC always knows if a nested-name-specifier is a type.

Basically, I've gotten the hang of recovering from missing 'typename' now, so I don't think it will be that hard to handle this without token substitution.

So if we added a little macro-like expansion for template arguments as the tokens get replayed could that potentially lead to a lot of simplification in Sema?

It sounds potentially quite light to implement. What's footprint impact of the Sema-based approach, is it worth my taking a look?

So if we added a little macro-like expansion for template arguments as the
tokens get replayed could that potentially lead to a lot of simplification
in Sema?

Maybe, but we might still want the complex sema bits for error recovery.

It sounds potentially quite light to implement. What's footprint impact of
the Sema-based approach, is it worth my taking a look?

The problem is that we wouldn't be able to diagnose missing typenames easily. All of our diagnostics designed for template authors would never fire, because we'd never build the dependent, pre-instantiation ASTs for inline methods.

By pursuing the AST + Sema approach, we can improve error recovery in mainline Clang when MSVC compatibility is disabled, and minimize divergence between clang with -fms-compatibility / -fdelayed-template-parsing and clang without.