Skip to content

Migrate x86_64-fortanix-unknown-sgx-lvirun-make test to rmake - #129055

Merged
bors merged 1 commit into
rust-lang:masterfrom
Oneirical:fortanix-fortification
Aug 23, 2024
Merged

Migrate x86_64-fortanix-unknown-sgx-lvirun-make test to rmake#129055
bors merged 1 commit into
rust-lang:masterfrom
Oneirical:fortanix-fortification

Conversation

@Oneirical

Copy link
Copy Markdown
Contributor

Part of #121876 and the associated Google Summer of Code project.

The final Makefile! Every Makefile test is now claimed.

This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in jobs.yml, but if there is a way to test it, please do.

Locally, on Linux, it compiles and panics at the llvm_filecheck part (if I replace the x86_64-fortanix-unknown-sgx with x86_64-unknown-linux-gnu, of course), which is expected.

For this reason, the Makefile and associated script have been kept, but with a leading underscore.

@rustbot

Copy link
Copy Markdown
Collaborator

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbotrustbot added A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 13, 2024
@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

@jieyouxu

jieyouxu commented Aug 14, 2024

Copy link
Copy Markdown
Member

I'm going to be real with you, I have absolutely no clue about sgx. So I'm going to request help from our sgx target maintainers after I take a look and try to guess what the test is trying to test.

Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
@workingjubilee

Copy link
Copy Markdown
Member

This is about verifying that LLVM cross-compiles SGX code with load value injection defenses.

It should be possible to make it work after building the cross-compile target.

@workingjubilee

Copy link
Copy Markdown
Member

This target is tier 2 so we should be building the cross-compilation artifacts every time anyways.

@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from 1f9cc33 to dc650f6CompareAugust 15, 2024 14:19
@workingjubilee

Copy link
Copy Markdown
Member

So: per policy, tests don't have to pass on SGX, a tier 2 target, in order to merge this. As this is an SGX-only test, we can just merge this through.

I do know, however, that the SGX maintainers do in fact run the CI regularly and add ignore-sgx, etc. and are reasonably responsive, usually.

@jethrogb@raoulstrackx@mzohreva Can we get your review?

@raoulstrackx

Copy link
Copy Markdown
Contributor

Thanks for tagging @workingjubilee ! I have a day off today, but I'll try to look at it on Monday.

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the port, I have several questions (that we will need help from the sgx maintainers).

Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
@jieyouxu

jieyouxu commented Aug 16, 2024

Copy link
Copy Markdown
Member

Thanks for tagging @workingjubilee ! I have a day off today, but I'll try to look at it on Monday.

@raoulstrackx Thank you very much for helping out! ❤️

@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from dc650f6 to aa15961CompareAugust 16, 2024 15:03
@rustbot

Copy link
Copy Markdown
Collaborator

The run-make-support library was changed

cc @jieyouxu

Comment threadsrc/tools/run-make-support/src/env.rs Outdated
Comment threadsrc/tools/run-make-support/src/env.rs Outdated
@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from aa15961 to df2ec7dCompareAugust 16, 2024 15:26

@raoulstrackxraoulstrackx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment this PR breaks CI. See comments. Can you please check it? You shouldn't have to have an SGX machine as the enclaves don't need to be executed.

Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs
@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch 3 times, most recently from 9e0ff38 to ffdde64CompareAugust 19, 2024 15:33
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from ffdde64 to b5a6a38CompareAugust 20, 2024 15:44
@raoulstrackx

Copy link
Copy Markdown
Contributor

There are other issues as well. Please apply:

diff --git a/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs b/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs index 84c292d019d..506b9424d39 100644 --- a/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs +++ b/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs @@ -41,13 +41,13 @@ fn main() { check("std::io::stdio::_print::[[:alnum:]]+", "print.with_frame_pointers.checks"); - check("st_plus_one_global_asm", "rust_plus_one_global_asm.checks"); + check("rust_plus_one_global_asm", "rust_plus_one_global_asm.checks"); - check("_plus_one_c", "cc_plus_one_c.checks"); - check("_plus_one_c_asm", "cc_plus_one_c_asm.checks"); - check("_plus_one_cxx", "cc_plus_one_cxx.checks"); - check("_plus_one_cxx_asm", "cc_plus_one_cxx_asm.checks"); - check("_plus_one_asm", "cc_plus_one_asm.checks"); + check("cc_plus_one_c", "cc_plus_one_c.checks"); + check("cc_plus_one_c_asm", "cc_plus_one_c_asm.checks"); + check("cc_plus_one_cxx", "cc_plus_one_cxx.checks"); + check("cc_plus_one_cxx_asm", "cc_plus_one_cxx_asm.checks"); + check("cc_plus_one_asm", "cc_plus_one_asm.checks"); check("cmake_plus_one_c", "cmake_plus_one_c.checks"); check("cmake_plus_one_c_asm", "cmake_plus_one_c_asm.checks"); @@ -66,9 +66,11 @@ fn check(func_re: &str, mut checks: &str) { .stdout_utf8(); let re = regex::Regex::new(&format!("[[:blank:]]+{func_re}")).unwrap(); let func = re.find_iter(&dump).map(|m| m.as_str().trim()).collect::<Vec<&str>>().join(","); + assert!(!func.is_empty()); + let dump = llvm_objdump() .input("enclave/target/x86_64-fortanix-unknown-sgx/debug/enclave") - .arg(&format!("--disassemble-symbols={func}")) + .args(&["--demangle", &format!("--disassemble-symbols={func}")]) .run() .stdout_utf8(); let dump = dump.as_bytes();

@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from b5a6a38 to 6a9e9fcCompareAugust 21, 2024 14:27
@Oneirical

Oneirical commented Aug 21, 2024

Copy link
Copy Markdown
ContributorAuthor

There are other issues as well. Please apply:

Oh dear, I should have watched where my editor's multiple cursors were going. Thank you so much once again! Changes applied.

@raoulstrackxraoulstrackx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the tests all pass. :)

Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/_Makefile Outdated
Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/_script.sh Outdated
@Oneirical
Oneiricalforce-pushed the fortanix-fortification branch from 6a9e9fc to e276d22CompareAugust 22, 2024 14:22
@Oneirical

Oneirical commented Aug 22, 2024

Copy link
Copy Markdown
ContributorAuthor

Now the tests all pass. :)

I deleted both Makefile and script.sh as they are no longer needed, let me know if you'd like to keep them.

Thank you so much for all your help!! This would have been much harder without a fortanix-rust maintainer.

@jieyouxu

Copy link
Copy Markdown
Member

Now the tests all pass. :)

@raoulstrackx thank you so much for helping to review and test 💚

@jieyouxu

Copy link
Copy Markdown
Member

I deleted both Makefile and script.sh as they are no longer needed, let me know if you'd like to keep them.

They're still in the change history, and now the rmake.rs version reimplements them, so we shouldn't keep them.

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this LGTM as well. Feel free to r=me after PR CI is green.

Comment threadtests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated
@jieyouxu

Copy link
Copy Markdown
Member

@bors delegate+

@bors

bors commented Aug 22, 2024

Copy link
Copy Markdown
Collaborator

✌️ @Oneirical, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, please make that change, then do @bors r=@jieyouxu

@Oneirical

Copy link
Copy Markdown
ContributorAuthor

@bors r=@jieyouxu

@bors

bors commented Aug 22, 2024

Copy link
Copy Markdown
Collaborator

📌 Commit e276d22 has been approved by jieyouxu

It is now in the queue for this repository.

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 22, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
…iaskrgr
Rollup of 8 pull requests
Successful merges:
- rust-lang#127623 (fix: fs::remove_dir_all: treat internal ENOENT as success)
- rust-lang#128876 (Ship MinGW-w64 runtime DLLs along with `rust-lld.exe` for `-pc-windows-gnu` targets)
- rust-lang#129055 (Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake)
- rust-lang#129386 (Use a LocalDefId in ResolvedArg.)
- rust-lang#129400 (Update `compiler_builtins` to `0.1.120`)
- rust-lang#129414 (Fix extern crates not being hidden with `doc(hidden)`)
- rust-lang#129417 (Don't trigger refinement lint if predicates reference errors)
- rust-lang#129433 (Fix a missing import in a doc in run-make-support)
r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
…iaskrgr
Rollup of 8 pull requests
Successful merges:
- rust-lang#127623 (fix: fs::remove_dir_all: treat internal ENOENT as success)
- rust-lang#128876 (Ship MinGW-w64 runtime DLLs along with `rust-lld.exe` for `-pc-windows-gnu` targets)
- rust-lang#129055 (Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake)
- rust-lang#129386 (Use a LocalDefId in ResolvedArg.)
- rust-lang#129400 (Update `compiler_builtins` to `0.1.120`)
- rust-lang#129414 (Fix extern crates not being hidden with `doc(hidden)`)
- rust-lang#129417 (Don't trigger refinement lint if predicates reference errors)
- rust-lang#129433 (Fix a missing import in a doc in run-make-support)
r? `@ghost`
`@rustbot` modify labels: rollup
@bors
bors merged commit fe87433 into rust-lang:masterAug 23, 2024
@rustbotrustbot added this to the 1.82.0 milestone Aug 23, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
Rollup merge of rust-lang#129055 - Oneirical:fortanix-fortification, r=jieyouxu
Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake
Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
The final Makefile! Every Makefile test is now claimed.
This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in `jobs.yml`, but if there is a way to test it, please do.
Locally, on Linux, it compiles and panics at the `llvm_filecheck` part (if I replace the `x86_64-fortanix-unknown-sgx` with `x86_64-unknown-linux-gnu`, of course), which is expected.
For this reason, the Makefile and associated script have been kept, but with a leading underscore.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-run-makeArea: port run-make Makefiles to rmake.rsA-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants

@Oneirical@rustbot@jieyouxu@workingjubilee@raoulstrackx@bors