Use a proc macro to declare preallocated symbols

[Zoxc]

r? @petrochenkov

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:01504910:start=1554252951726268766,finish=1554252952603197523,duration=876928757
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---
[00:07:07] Compiling rustc_macros v0.1.0 (/checkout/src/librustc_macros)
[00:07:07] error[E0583]: file not found for module `symbols`
[00:07:07] --> src/librustc_macros/src/lib.rs:12:5
[00:07:07] |
[00:07:07] 12 | mod symbols;
[00:07:07] |
[00:07:07] |
[00:07:07] = help: name the file either symbols.rs or symbols/mod.rs inside the directory "src/librustc_macros/src"
[00:07:07] error: aborting due to previous error
[00:07:07] [00:07:07] For more information about this error, try `rustc --explain E0583`.
[00:07:07] error: Could not compile `rustc_macros`.
---
travis_time:end:017f782b:start=1554253391329039133,finish=1554253391337608513,duration=8569380
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:06d40b80
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:00116596
travis_time:start:00116596
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Mark-Simulacrum]

@bors try for perf

[bors]

⌛ Trying commit e76ebf77a364229542ea4a97bd76f0f1e74ad8d1 with merge 61412ac6c97f93955672e1bafe3f9cd5e1edbab8...

[bors]

☀️ Try build successful - checks-travis
Build commit: 61412ac6c97f93955672e1bafe3f9cd5e1edbab8

[Zoxc]

@rust-timer build 61412ac6c97f93955672e1bafe3f9cd5e1edbab8

[rust-timer]

Success: Queued 61412ac6c97f93955672e1bafe3f9cd5e1edbab8 with parent 7641873, comparison URL.

[rust-timer]

Finished benchmarking try commit 61412ac6c97f93955672e1bafe3f9cd5e1edbab8

[petrochenkov]

Random thoughts:

• Proc macros are a heavy hammer, I'd really prefer to use them only if macro_rules cannot do the job.
  In this case the only thing that macro_rules cannot do is counting (or it can, but with higher complexity than a proc macro).
  For the symbols pre-interned so far just writing the numbers manually would be preferable to introducing a proc macro, but I can imagine how this can become an issue if more symbols is pre-interned (e.g. all built-in types u8, u16, etc).
  (FWIW, "Go To Definition" on the keywords doesn't work even with declarative macros in my editor, and mod keywords is still searchable as a text, so using a proc macro doesn't regress at least this part.)
• Perf results are mostly noise, so not doing this and continuing working with strs is a real alternative.
• In some sense this gives libsyntax_pos the knowledge about entities from entirely unrelated crates, that's probably not too bad given that they are are parts of the language (grammars of various attributes), like keywords.
• Names of built-in attributes are one of the primary sources of strs right now, but they cannot be pre-interned without either duplicating that huge list of attributes or moving all the knowledge about attributes into syntax_pos (both variants don't look especially good).
• I actually wanted to get rid of struct Keyword and move keywords to the scheme that's used for symbols in this PR - just a const Match: Symbol = .... That's probably a material for another PR.

cc @rust-lang/compiler
Any opinions on casual use of proc macros in the compiler?

[Zoxc]

• Proc macros are a heavy hammer, I'd really prefer to use them only if macro_rules cannot do the job.
  In this case the only thing that macro_rules cannot do is counting (or it can, but with higher complexity than a proc macro).
  For the symbols pre-interned so far just writing the numbers manually would be preferable to introducing a proc macro, but I can imagine how this can become an issue if more symbols is pre-interned (e.g. all built-in types u8, u16, etc).

Yeah, counting with macro_rules is annoying and could quite possibly be slow if this list of symbols is extended. Also I wouldn't want to do this with manual counting.

(FWIW, "Go To Definition" on the keywords doesn't work even with declarative macros in my editor, and mod keywords is still searchable as a text, so using a proc macro doesn't regress at least this part.)

I made the keywords and symbols modules regular non-macro modules, Racer is still confused though.

• Perf results are mostly noise, so not doing this and continuing working with strs is a real alternative.

Perf results are pretty irrelevant since this isn't applied to anything yet.

[petrochenkov]

👍

[petrochenkov]

@bors r+

[bors]

📌 Commit 30a86cf has been approved by petrochenkov

[bors]

🌲 The tree is currently closed for pull requests below priority 15, this pull request will be tested once the tree is reopened

[Centril]

@bors r-

[Zoxc]

@bors r=petrochenkov

[bors]

📌 Commit f598091 has been approved by petrochenkov

[bors]

⌛ Testing commit f598091 with merge 8a09e37289afb0f45e088bcb70465070a68a3c5c...

[bors]

💔 Test failed - checks-travis

[rust-highfive]

The job arm-android of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[01:44:21] test string::test_str_clear ... ok
[01:44:21] test string::test_str_truncate ... ok
[01:44:21] test string::test_str_truncate_invalid_len ... ok
[01:44:21] test string::test_str_truncate_split_codepoint ... ok
[01:44:21] died due to signal 11
[01:44:21] [01:44:21] [01:44:21] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "arm-linux-androideabi" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "-p" "alloc" "--"
[01:44:21] expected success, got: exit code: 3
---
travis_time:end:0900cbc0:start=1555312243973300663,finish=1555312243980997977,duration=7697314
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:01bf6408
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1099af60
travis_time:start:1099af60
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:18db65b4
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Zoxc]

@rust-lang/infra Is this a spurious segfault?

[kennytm]

Yes.

@bors retry #55861

[bors]

⌛ Testing commit f598091 with merge fcf850f...

[bors]

☀️ Test successful - checks-travis, status-appveyor
Approved by: petrochenkov
Pushing fcf850f to master...