Update `Environment.ProcessorCount` on Windows to take into account the processor affinity mask by kouvel · Pull Request #45943 · dotnet/runtime

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel · 2020-12-11T03:33:15Z

Similarly to cases on Unixes where sched_getaffinity is available
If GCCpuGroup and Thread_UseAllCpuGroups are both enabled, I'm not sure if the CPUGroupInfo count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it
Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count and perf is much worse in CPU-affinitized cases compared to the native thread pool
This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask
After this change, in affinitized cases on Windows, perf-wise where this difference matters, the behavior perf-wise is closer to that on Linux and closer to what is currently expected:
- The portable thread pool uses the same worker thread count as the native thread pool
- Environment.ProcessorCount returns the number of processors the process is affinitized to, which may be less than it would have returned before, similarly to Linux

Breaking change issue: #47427

… processor affinity mask - Similarly to cases on Unixes where sched_getaffinity is available - If `GCCpuGroup` and `Thread_UseAllCpuGroups` are both enabled, I'm not sure if the `CPUGroupInfo` count of active processors takes affinity into account as the docs are not clear, for now I'm not modifying that path until I can verify it - Otherwise, a process that is started with a specific processor affinity mask still shows full CPU count - This is one of the differences in the portable managed thread pool implementation, which relies on Environment.ProcessorCount, as opposed to the native thread pool, which uses the affinity mask - After this change, in affinitized cases on Windows the behavior is consistent perf-wise with Linux in similar situations: - The portable thread pool uses the same worker thread count as the native thread pool - `Environment.ProcessorCount` returns the number of processors the the process is affinitized to, which may be less than it would have returned before

pgrawehr · 2020-12-17T11:43:10Z

I don't think this is a good idea. After this change, how would one obtain the processor count, for instance to check whether an affinity mask is applied? For instance, I have a piece of code that uses this property and compares it to the affinity mask to make sure the application is allowed to run on all CPUs.

jkotas · 2020-12-17T14:38:10Z

@pgrawehr The number of processors that the application can run on can be limited by number of mechanism. Process affinity, job control on Windows and cgroups on Linux are the most common mechanisms. We have found through trial-and-failure that returning the restricted number of processors from this property is the most appropriate behavior that makes most callers work well, and it is what we do on non-Windows systems already.

After this change, how would one obtain the processor count

You can PInvoke the appropriate OS APIs.

@kouvel I agree that this is a good change to make, but it needs to be marked as breaking change.

pgrawehr · 2020-12-17T15:04:41Z

You can PInvoke the appropriate OS APIs.

I think that's bad, because it's again OS dependent. The properties and methods in the Environment class are just there so the programmer doesn't need to use any low-level PInvoke calls for common system properties. Maybe adding a separate property would be nicer.

janvorli

LGTM, thank you!

danmoseley · 2020-12-17T17:56:02Z

@pgrawehr you could make an API proposal, with the template.

stephentoub · 2021-01-09T18:41:44Z

@kouvel, can this be merged?

kouvel · 2021-01-09T18:43:12Z

I just need to file an issue and document some things for the breaking change process, will try to do that early next week.

stephentoub · 2021-01-09T18:44:22Z

Ok, cool.

danmoseley · 2021-01-25T21:09:48Z

@kouvel when you open that breaking change issue please remove the "needs breaking change doc" label

…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

Port of dotnet/runtime#45943 Delete some dead code

kouvel added the area-System.Threading label Dec 11, 2020

kouvel added this to the 6.0.0 milestone Dec 11, 2020

kouvel requested a review from janvorli December 11, 2020 03:33

kouvel self-assigned this Dec 11, 2020

janvorli approved these changes Dec 17, 2020
View reviewed changes

danmoseley added the breaking-change Issue or PR that represents a breaking API or functional change over a previous release. label Dec 17, 2020

ghost added the needs-breaking-change-doc-created Breaking changes need an issue opened with https://github.com/dotnet/docs/issues/new?template=dotnet label Dec 17, 2020

pgrawehr mentioned this pull request Dec 21, 2020
Add Environment.TotalProcessorCount or similar API, after semantics of Environment.ProcessorCount has changed #46283
Closed

stephentoub reviewed Jan 4, 2021
View reviewed changes

Comment threadsrc/coreclr/vm/eeconfig.cpp

kouvel mentioned this pull request Jan 25, 2021
[Breaking change] Environment.ProcessorCount on Windows takes processor affinity into account #47427
Closed

kouvel merged commit e42de0e into dotnet:masterJan 25, 2021

kouvel deleted the TpProcessAffinity branch January 25, 2021 20:21

kouvel removed the needs-breaking-change-doc-created Breaking changes need an issue opened with https://github.com/dotnet/docs/issues/new?template=dotnet label Jan 25, 2021

jkotas added a commit to jkotas/runtimelab that referenced this pull request Jan 27, 2021
Update Environment.ProcessorCount on Windows to take into account t…
0eb8de6
…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

jkotas added a commit to jkotas/runtimelab that referenced this pull request Jan 27, 2021
Update Environment.ProcessorCount on Windows to take into account t…
75662a0
…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

jkotas added a commit to jkotas/runtimelab that referenced this pull request Jan 27, 2021
Update Environment.ProcessorCount on Windows to take into account t…
b0cbf15
…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

jkotas added a commit to jkotas/runtimelab that referenced this pull request Jan 27, 2021
Update Environment.ProcessorCount on Windows to take into account t…
8521230
…he processor affinity mask Port of dotnet/runtime#45943 Delete some dead code

MichalStrehovsky pushed a commit to dotnet/runtimelab that referenced this pull request Jan 28, 2021
Update Environment.ProcessorCount - portdotnet/runtime#45943(#598)
e28a392
Port of dotnet/runtime#45943 Delete some dead code

ghost locked as resolved and limited conversation to collaborators Feb 24, 2021