Add a pointers interface to transparently handle user/kernel split by jaybosamiya-ms · Pull Request #11 · microsoft/litebox

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:10:24Z

This PR introduces a RawPointerProvider trait interface to the platform which the platform can use to support a user/kernel pointer split (i.e., supporting a copy-to-user/copy-to-kernel functionality basically transparently). The PR also implements "transparent pointers" for use in pure user-land platforms that don't have a meaningful user/kernel split. Finally, it updates relevant parts of the codebase to account for this new API.

Morally, a RawConstPointer<T> is a *const Tin userland, while a bare *const T (without anything else specified) is intended to be a kernel pointer (again, only in platforms where this distinction makes sense; in platforms where it doesn't, both are identical).

Closes#7

Jay Bosamiya (Microsoft) (jaybosamiya-ms) · 2025-02-13T23:13:13Z

Maybe I should update the SemverChecks CI to be "warn but don't mark an X" for now? I think it is a very useful thing to have in the CI, since it points out important possible breakages, but currently since we are working on the absolutely early pre-release, we don't actually want it to be complaining so much. EDIT: I have updated the semver checks with this in #12

Weiteng Chen (CvvT)

LGTM!

Weiteng Chen (CvvT) · 2025-02-15T01:07:28Z

+ ///
+ /// The pointer (and underlying memory for the value at the offset) should remain valid and
+ /// unchanged for the entirety of the lifetime that the borrow (if any) is made.
+ unsafe fn read_at_offset<'a>(self, count: isize) -> Option<alloc::borrow::Cow<'a, T>>;


Jay Bosamiya (Microsoft) (@jaybosamiya-ms) Should we add an interface is_null? Sometimes we need to distinguish between null pointer and unmapped memory.

This interface is meant to be used mostly within the shim layer. Maybe I am missing a specific use case where the shim would need to be aware of this difference. Happy to add an is_null to the trait if there is such a use case, but at least right now I am unable to think of a case where this matters (at the shim layer; I would completely understand it mattering at the platform or lower layers, but those layers are fully aware of the pointer details, so can do extra checks internally if needed).

Jay Bosamiya (Microsoft) (jaybosamiya-ms) requested a review from Weidong Cui (wdcui)February 13, 2025 23:10

Jay Bosamiya (Microsoft) (jaybosamiya-ms)force-pushed the jayb/push-uovknntlmrwn branch from bc40235 to cdbcdcdCompare February 14, 2025 00:29

Jay Bosamiya (Microsoft) (jaybosamiya-ms) added 5 commits February 13, 2025 16:32

Add a raw readable pointers interface
a9e7a2f

Add a raw writable pointers interface
8fd9067

Implement transparent raw pointers useful for user-mode platforms
b81127e

Add raw pointers to the requirements of a full platform
4a54295

Update shim to use the pointer API
ac21d8a

Jay Bosamiya (Microsoft) (jaybosamiya-ms)force-pushed the jayb/push-uovknntlmrwn branch from cdbcdcd to ac21d8aCompare February 14, 2025 00:33

Weidong Cui (wdcui) approved these changes Feb 14, 2025
View reviewed changes

Comment threadlitebox/src/platform/mod.rs

Jay Bosamiya (Microsoft) (jaybosamiya-ms) requested a review from Weiteng Chen (CvvT)February 14, 2025 21:23

Weiteng Chen (CvvT) approved these changes Feb 14, 2025
View reviewed changes

Jay Bosamiya (Microsoft) (jaybosamiya-ms) merged commit 2ef1ca5 into mainFeb 14, 2025

Jay Bosamiya (Microsoft) (jaybosamiya-ms) deleted the jayb/push-uovknntlmrwn branch February 14, 2025 23:59

Weiteng Chen (CvvT) reviewed Feb 15, 2025
View reviewed changes

Jay Bosamiya (Microsoft) (jaybosamiya-ms) mentioned this pull request Feb 22, 2025
Where should we check untrusted pointers into memory? #6
Open