Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kunalspathak merged 21 commits into from
Jun 11, 2024
Merged

Handle more than 64 registers - Part 5 #103188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4f81f69
first cut
kunalspathak Jun 7, 2024
c8f9b15
make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*
kunalspathak Jun 7, 2024
e150ec5
make clrjit_universal_arm_x64 build
kunalspathak Jun 7, 2024
f7e759f
make everything else build
kunalspathak Jun 7, 2024
f887a6d
fix some static asserts
kunalspathak Jun 8, 2024
c50b029
jit format
kunalspathak Jun 8, 2024
a3bea88
minor static assert fix
kunalspathak Jun 8, 2024
2e2d48c
Remove RegMaskTP_NONE
kunalspathak Jun 8, 2024
bbf7da1
fix some build errors
kunalspathak Jun 8, 2024
80b658c
jit format
kunalspathak Jun 8, 2024
8ee0adc
some more fixes
kunalspathak Jun 8, 2024
1534499
Use regMaskTP for kill RefPositions
kunalspathak Jun 8, 2024
73d011b
Merge remote-tracking branch 'origin/main' into part5
kunalspathak Jun 8, 2024
f5ceca1
fix a typo
kunalspathak Jun 9, 2024
21e3c2f
Fix build errors for riscv64/loongarch64
kunalspathak Jun 10, 2024
b68f2d3
try to fix gcc errors
kunalspathak Jun 10, 2024
7f5a542
jit format
kunalspathak Jun 10, 2024
3d02d69
Revert "try to fix gcc errors"
kunalspathak Jun 10, 2024
f1b00a6
fix gcc build failure
kunalspathak Jun 10, 2024
491da62
minor code cleanup
kunalspathak Jun 10, 2024
b64e835
review comments
kunalspathak Jun 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/coreclr/jit/codegenxarch.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
// -saved bool for synchronized methods

// slots for ret address + FP + EnC callee-saves
int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
{
Expand Down
9 changes: 5 additions & 4 deletions src/coreclr/jit/compiler.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
return reg;
}

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
{
return regMask;
}

private:
regNumber reg;
_regMask_enum regMask;
regNumber reg;
regMaskTP regMask;
};

VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
// On these platforms we assume the register that the target is
// passed in is preserved by the validator and take care to get the
// target from the register for the call (even in debug mode).
static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==

@jakobbotschjakobbotschJun 10, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).

@kunalspathakkunalspathakJun 10, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.

RBM_NONE);
if (JitConfig.JitForceControlFlowGuard())
return true;

Expand Down
4 changes: 2 additions & 2 deletions src/coreclr/jit/emit.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
callDsc* call;

#ifdef JIT32_GCENCODER
unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

// The JIT32 GCInfo encoder allows us to (as the comment previously here said):
// "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
// of callee-saved registers only).
for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
{
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
if (emitThisGCrefRegs & calleeSavedRbm)
{
gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down
6 changes: 3 additions & 3 deletions src/coreclr/jit/emitarm.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
}
else
{
hasPC = (imm & RBM_PC) != 0;
hasLR = (imm & RBM_LR) != 0;
imm &= ~(RBM_PC | RBM_LR);
hasPC = (imm & SRBM_PC) != 0;
hasLR = (imm & SRBM_LR) != 0;
imm &= ~(SRBM_PC | SRBM_LR);
Comment on lines +6870 to +6872

@jakobbotschjakobbotschJun 10, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?

@kunalspathakkunalspathakJun 10, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.

}

regNumber reg = REG_R0;
Expand Down
6 changes: 3 additions & 3 deletions src/coreclr/jit/emitinl.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
unsigned encodeMask;

#ifdef TARGET_X86
assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
NYI("unknown target");
#endif

return regmask;
return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down
10 changes: 6 additions & 4 deletions src/coreclr/jit/gcencode.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

// Other than that, we just have to deal with the regmasks.
regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
{
// This is a true call site.

regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
regMaskSmall gcrefRegMask =
genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
regMaskSmall byrefRegMask =
genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down
108 changes: 53 additions & 55 deletions src/coreclr/jit/lsra.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
return availableIntRegs & RBM_BYTE_REGS;
return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
return (availableFloatRegs & RBM_LOWFLOAT);
return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
RefPosition* kill = nextKill;
while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
{
#ifdef HAS_MORE_THAN_64_REGISTERS
if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
{
nextLocation = kill->nodeLocation;
break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
}
else
{
#ifdef TARGET_AMD64
return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
}
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
(RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
(RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
(RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
case LSRA_LIMIT_CALLEE:
if (!compiler->opts.compDbgEnC)
{
mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
minRegCount);
}
break;

case LSRA_LIMIT_CALLER:
{
#ifdef TARGET_XARCH
mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
minRegCount);
#else
mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
}
break;

case LSRA_LIMIT_SMALL_SET:
if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
{
mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
mask = getConstrainedRegMask(refPosition, regType, mask,
LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
}
else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
{
mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
mask = getConstrainedRegMask(refPosition, regType, mask,
LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
}
break;

#if defined(TARGET_AMD64)
case LSRA_LIMIT_UPPER_SIMD_SET:
if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
{
mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
mask = getConstrainedRegMask(refPosition, regType, mask,
LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
}
break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
// Note: one known reason why we exclude LR is because NativeAOT has dependency on not
// using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
// Once that is addressed, we may consider allowing LR in availableIntRegs.
availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
availableIntRegs =
(RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
availableIntRegs &= ~RBM_FPBASE;
availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
availableFloatRegs = RBM_ALLFLOAT;
availableDoubleRegs = RBM_ALLDOUBLE;
availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
if (compiler->opts.compDbgEnC)
{
// When the EnC option is set we have an exact set of registers that we always save
// that are also available in future versions.
availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
if (compiler->canUseEvexEncoding())
{
availableFloatRegs |= RBM_HIGHFLOAT;
availableDoubleRegs |= RBM_HIGHFLOAT;
availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
}
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
SingleTypeRegSet removeMask = RBM_NONE;
if (frameType == FT_EBP_FRAME)
{
removeMask |= RBM_FPBASE;
removeMask |= RBM_FPBASE.GetIntRegSet();
}

compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
assert(REG_OPT_RSVD != REG_FP);
JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
removeMask |= RBM_OPT_RSVD;
removeMask |= RBM_OPT_RSVD.GetIntRegSet();
}
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
RefPosition* nextKill = killRefPosition->nextRefPosition;

regMaskTP killedRegs = killRefPosition->registerAssignment;
regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
while (killedRegs.IsNonEmpty())
{
regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
updateNextFixedRef(regRecord, regNextRefPos, nextKill);
}

regsBusyUntilKill &= ~killRefPosition->registerAssignment;
regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
killRefPosition->registerAssignment));
killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
{
// Exclude any doubles for which the odd half isn't in freeRegs,
// and restrict down to just the even part of the even/odd pair.
freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
}
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
else
{
// Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
if ((freeRegs & RBM_CALLEE_TRASH) != 0)
if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
{
#ifdef TARGET_XARCH
freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
freeRegs &= RBM_CALLEE_TRASH;
#endif
}

regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
// clause below creates a mask to do this.
if (currentInterval->registerType == TYP_DOUBLE)
{
candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
}
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
// clause below creates a mask to do this.
if (currentInterval->registerType == TYP_DOUBLE)
{
candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
}
#endif // TARGET_ARM

Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Handle more than 64 registers - Part 5#103188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kunalspathak
merged 21 commits into

from

Jun 11, 2024
 Merged


Handle more than 64 registers - Part 5
#103188
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


21 commits


Select commit
Hold shift + click to select a range
4f81f69

first cut


kunalspathak Jun 7, 2024
c8f9b15

make clrjit/clrjit_universal_arm64_x64 working, introduce SRBM_*


kunalspathak Jun 7, 2024
e150ec5

make clrjit_universal_arm_x64 build


kunalspathak Jun 7, 2024
f7e759f

make everything else build


kunalspathak Jun 7, 2024
f887a6d

fix some static asserts


kunalspathak Jun 8, 2024
c50b029

jit format


kunalspathak Jun 8, 2024
a3bea88

minor static assert fix


kunalspathak Jun 8, 2024
2e2d48c

Remove RegMaskTP_NONE


kunalspathak Jun 8, 2024
bbf7da1

fix some build errors


kunalspathak Jun 8, 2024
80b658c

jit format


kunalspathak Jun 8, 2024
8ee0adc

some more fixes


kunalspathak Jun 8, 2024
1534499

Use regMaskTP for kill RefPositions


kunalspathak Jun 8, 2024
73d011b

Merge remote-tracking branch 'origin/main' into part5


kunalspathak Jun 8, 2024
f5ceca1

fix a typo


kunalspathak Jun 9, 2024
21e3c2f

Fix build errors for riscv64/loongarch64


kunalspathak Jun 10, 2024
b68f2d3

try to fix gcc errors


kunalspathak Jun 10, 2024
7f5a542

jit format


kunalspathak Jun 10, 2024
3d02d69

Revert "try to fix gcc errors"


kunalspathak Jun 10, 2024
f1b00a6

fix gcc build failure


kunalspathak Jun 10, 2024
491da62

minor code cleanup


kunalspathak Jun 10, 2024
b64e835

review comments


kunalspathak Jun 10, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/coreclr/jit/codegenxarch.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8841,7 +8841,7 @@ void CodeGen::genCreateAndStoreGCInfoX64(unsigned codeSize, unsigned prologSize
 // -saved bool for synchronized methods

 // slots for ret address + FP + EnC callee-saves
 int preservedAreaSize = (2 + genCountBits((uint64_t)RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;
 int preservedAreaSize = (2 + genCountBits(RBM_ENC_CALLEE_SAVED)) * REGSIZE_BYTES;

 if (compiler->info.compFlags & CORINFO_FLG_SYNCH)
 {
Expand Down

9 changes: 5 additions & 4 deletions
src/coreclr/jit/compiler.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8317,14 +8317,14 @@ class Compiler
 return reg;
 }

_regMask_enum GetRegMask() const
regMaskTP GetRegMask() const
 {
 return regMask;
 }

 private:
 regNumber reg;
_regMask_enum regMask;
 regNumber reg;
regMaskTP regMask;
 };

 VirtualStubParamInfo* virtualStubParamInfo;
Expand DownExpand Up@@ -9851,7 +9851,8 @@ class Compiler
 // On these platforms we assume the register that the target is
 // passed in is preserved by the validator and take care to get the
 // target from the register for the call (even in debug mode).
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & (1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) == 0);
 static_assert_no_msg((RBM_VALIDATE_INDIRECT_CALL_TRASH & regMaskTP(1 << REG_VALIDATE_INDIRECT_CALL_ADDR)) ==
@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make IsRegNumInMaskconstexpr to make this a bit more natural (feel free to do as part of a follow-up).
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing that, but as a cascade effect I need to make genSingleTypeRegSet, getRegForType(), etc. as constexpr which I am not sure is worth doing for just an assert. Will pass this on for now.
 RBM_NONE);
 if (JitConfig.JitForceControlFlowGuard())
 return true;

Expand Down

4 changes: 2 additions & 2 deletions
src/coreclr/jit/emit.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8787,7 +8787,7 @@ void emitter::emitRecordGCcall(BYTE* codePos, unsigned char callInstrSize)
 callDsc* call;

#ifdef JIT32_GCENCODER
 unsigned regs = (unsigned)(emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET;
 unsigned regs = (unsigned)((emitThisGCrefRegs | emitThisByrefRegs) & ~RBM_INTRET).GetIntRegSet();

 // The JIT32 GCInfo encoder allows us to (as the comment previously here said):
 // "Bail if this is a totally boring call", but the GCInfoEncoder/Decoder interface
Expand DownExpand Up@@ -10062,7 +10062,7 @@ void emitter::emitStackPopLargeStk(BYTE* addr, bool isCall, unsigned char callIn
 // of callee-saved registers only).
 for (unsigned calleeSavedRegIdx = 0; calleeSavedRegIdx < CNT_CALL_GC_REGS; calleeSavedRegIdx++)
 {
regMaskSmall calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
regMaskTP calleeSavedRbm = raRbmCalleeSaveOrder[calleeSavedRegIdx];
 if (emitThisGCrefRegs & calleeSavedRbm)
 {
 gcrefRegs |= (1 << calleeSavedRegIdx);
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitarm.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6867,9 +6867,9 @@ void emitter::emitDispRegmask(int imm, bool encodedPC_LR)
 }
 else
 {
 hasPC = (imm & RBM_PC) != 0;
 hasLR = (imm & RBM_LR) != 0;
 imm &= ~(RBM_PC | RBM_LR);
 hasPC = (imm & SRBM_PC) != 0;
 hasLR = (imm & SRBM_LR) != 0;
 imm &= ~(SRBM_PC | SRBM_LR);

Comment on lines

+6870
 to 
+6872

@jakobbotschjakobbotschJun 10, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means there is (and was) an invariant on the exact indices defined for these registers?
@kunalspathakkunalspathakJun 10, 2024

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably.
 }

 regNumber reg = REG_R0;
Expand Down

6 changes: 3 additions & 3 deletions
src/coreclr/jit/emitinl.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -403,8 +403,8 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const

/*static*/ inline unsigned emitter::emitDecodeCallGCregs(instrDesc* id)
{
unsigned regmask = 0;
 unsigned encodeMask;
regMaskTP regmask = RBM_NONE;
 unsigned encodeMask;

#ifdef TARGET_X86
 assert(REGNUM_BITS >= 3);
Expand DownExpand Up@@ -568,7 +568,7 @@ inline ssize_t emitter::emitGetInsAmdAny(const instrDesc* id) const
 NYI("unknown target");
#endif

 return regmask;
 return (unsigned int)regmask.getLow();
}

#ifdef TARGET_XARCH
Expand Down

10 changes: 6 additions & 4 deletions
src/coreclr/jit/gcencode.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -4471,8 +4471,8 @@ void GCInfo::gcMakeRegPtrTable(
 assert(call->u1.cdArgMask == 0 && call->cdArgCnt == 0);

 // Other than that, we just have to deal with the regmasks.
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS;
 regMaskSmall gcrefRegMask = call->cdGCrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();
 regMaskSmall byrefRegMask = call->cdByrefRegs & RBM_CALL_GC_REGS.GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand DownExpand Up@@ -4558,9 +4558,11 @@ void GCInfo::gcMakeRegPtrTable(
 {
 // This is a true call site.

 regMaskSmall gcrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs);
 regMaskSmall gcrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallGCrefRegs).GetIntRegSet();

 regMaskSmall byrefRegMask = genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs);
 regMaskSmall byrefRegMask =
 genRegMaskFromCalleeSavedMask(genRegPtrTemp->rpdCallByrefRegs).GetIntRegSet();

 assert((gcrefRegMask & byrefRegMask) == 0);

Expand Down

108 changes: 53 additions & 55 deletions
src/coreclr/jit/lsra.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@ SingleTypeRegSet LinearScan::allRegs(RegisterType rt)
SingleTypeRegSet LinearScan::allByteRegs()
{
#ifdef TARGET_X86
 return availableIntRegs & RBM_BYTE_REGS;
 return availableIntRegs & RBM_BYTE_REGS.GetIntRegSet();
#else
 return availableIntRegs;
#endif
Expand All@@ -265,7 +265,7 @@ SingleTypeRegSet LinearScan::allSIMDRegs()
SingleTypeRegSet LinearScan::lowSIMDRegs()
{
#if defined(TARGET_AMD64)
 return (availableFloatRegs & RBM_LOWFLOAT);
 return (availableFloatRegs & RBM_LOWFLOAT.GetFloatRegSet());
#else
 return availableFloatRegs;
#endif
Expand All@@ -278,7 +278,11 @@ void LinearScan::updateNextFixedRef(RegRecord* regRecord, RefPosition* nextRefPo
 RefPosition* kill = nextKill;
 while ((kill != nullptr) && (kill->nodeLocation < nextLocation))
 {
#ifdef HAS_MORE_THAN_64_REGISTERS
 if (kill->killRegisterAssignment.IsRegNumInMask(regRecord->regNum))
#else
 if ((kill->registerAssignment & genSingleTypeRegMask(regRecord->regNum)) != RBM_NONE)
#endif
 {
 nextLocation = kill->nodeLocation;
 break;
Expand DownExpand Up@@ -449,11 +453,7 @@ SingleTypeRegSet LinearScan::internalFloatRegCandidates()
 }
 else
 {
#ifdef TARGET_AMD64
 return RBM_FLT_CALLEE_TRASH.GetFloatRegSet();
#else
 return RBM_FLT_CALLEE_TRASH;
#endif // TARGET_AMD64
 }
}

Expand DownExpand Up@@ -526,34 +526,32 @@ SingleTypeRegSet LinearScan::getConstrainedRegMask(RefPosition* refPosition,
#if defined(TARGET_AMD64)
#ifdef UNIX_AMD64_ABI
// On System V the RDI and RSI are not callee saved. Use R12 ans R13 as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_R12 | RBM_R13);
#else // !UNIX_AMD64_ABI
// On Windows Amd64 use the RDI and RSI as callee saved registers.
static const SingleTypeRegSet LsraLimitSmallIntSet =
 (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EBX | RBM_ETW_FRAMED_EBP | RBM_ESI | RBM_EDI);
#endif // !UNIX_AMD64_ABI
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const SingleTypeRegSet LsraLimitUpperSimdSet =
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitUpperSimdSet =
 (RBM_XMM16 | RBM_XMM17 | RBM_XMM18 | RBM_XMM19 | RBM_XMM20 | RBM_XMM21 | RBM_XMM22 | RBM_XMM23 | RBM_XMM24 |
 RBM_XMM25 | RBM_XMM26 | RBM_XMM27 | RBM_XMM28 | RBM_XMM29 | RBM_XMM30 | RBM_XMM31);
#elif defined(TARGET_ARM)
// On ARM, we may need two registers to set up the target register for a virtual call, so we need
// to have at least the maximum number of arg registers, plus 2.
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R3 | RBM_R4 | RBM_R5);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F16 | RBM_F17);
#elif defined(TARGET_ARM64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_R0 | RBM_R1 | RBM_R2 | RBM_R19 | RBM_R20);
static const regMaskTP LsraLimitSmallFPSet = (RBM_V0 | RBM_V1 | RBM_V2 | RBM_V8 | RBM_V9);
#elif defined(TARGET_X86)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
static const regMaskTP LsraLimitSmallIntSet = (RBM_EAX | RBM_ECX | RBM_EDI);
static const regMaskTP LsraLimitSmallFPSet = (RBM_XMM0 | RBM_XMM1 | RBM_XMM2 | RBM_XMM6 | RBM_XMM7);
#elif defined(TARGET_LOONGARCH64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#elif defined(TARGET_RISCV64)
static const SingleTypeRegSet LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const SingleTypeRegSet LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
static const regMaskTP LsraLimitSmallIntSet = (RBM_T1 | RBM_T3 | RBM_A0 | RBM_A1 | RBM_T0);
static const regMaskTP LsraLimitSmallFPSet = (RBM_F0 | RBM_F1 | RBM_F2 | RBM_F8 | RBM_F9);
#else
#error Unsupported or unset target architecture
#endif // target
Expand DownExpand Up@@ -596,37 +594,37 @@ SingleTypeRegSet LinearScan::stressLimitRegs(RefPosition* refPosition, RegisterT
 case LSRA_LIMIT_CALLEE:
 if (!compiler->opts.compDbgEnC)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_SAVED.GetRegSetForType(regType),
 minRegCount);
 }
 break;

 case LSRA_LIMIT_CALLER:
 {
#ifdef TARGET_XARCH
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH.GetRegSetForType(regType),
 minRegCount);
#else
 mask = getConstrainedRegMask(refPosition, regType, mask, RBM_CALLEE_TRASH, minRegCount);
#endif // TARGET_AMD64
 }
 break;

 case LSRA_LIMIT_SMALL_SET:
 if ((mask & LsraLimitSmallIntSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallIntSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallIntSet.GetRegSetForType(regType), minRegCount);
 }
 else if ((mask & LsraLimitSmallFPSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitSmallFPSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitSmallFPSet.GetRegSetForType(regType), minRegCount);
 }
 break;

#if defined(TARGET_AMD64)
 case LSRA_LIMIT_UPPER_SIMD_SET:
 if ((mask & LsraLimitUpperSimdSet) != RBM_NONE)
 {
 mask = getConstrainedRegMask(refPosition, regType, mask, LsraLimitUpperSimdSet, minRegCount);
 mask = getConstrainedRegMask(refPosition, regType, mask,
 LsraLimitUpperSimdSet.GetRegSetForType(regType), minRegCount);
 }
 break;
#endif
Expand DownExpand Up@@ -847,39 +845,43 @@ LinearScan::LinearScan(Compiler* theCompiler)
 // Note: one known reason why we exclude LR is because NativeAOT has dependency on not
 // using LR as a GPR. See: https://github.com/dotnet/runtime/issues/101932
 // Once that is addressed, we may consider allowing LR in availableIntRegs.
 availableIntRegs = ((RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd.getLow()));
 availableIntRegs =
 (RBM_ALLINT & ~(RBM_PR | RBM_FP | RBM_LR) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#elif defined(TARGET_LOONGARCH64) || defined(TARGET_RISCV64)
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~(RBM_FP | RBM_RA) & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#else
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd.getLow());
 availableIntRegs = (RBM_ALLINT & ~compiler->codeGen->regSet.rsMaskResvd).GetIntRegSet();
#endif

#if ETW_EBP_FRAMED
 availableIntRegs &= ~RBM_FPBASE;
 availableIntRegs &= ~RBM_FPBASE.GetIntRegSet();
#endif // ETW_EBP_FRAMED

#ifdef TARGET_AMD64
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#else
 availableFloatRegs = RBM_ALLFLOAT;
 availableDoubleRegs = RBM_ALLDOUBLE;
 availableFloatRegs = RBM_ALLFLOAT.GetFloatRegSet();
 availableDoubleRegs = RBM_ALLDOUBLE.GetFloatRegSet();
#endif

#if defined(TARGET_XARCH)
#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK.GetPredicateRegSet();
#elif defined(TARGET_ARM64)
 availableMaskRegs = RBM_ALLMASK;
#endif

#if defined(TARGET_AMD64) || defined(TARGET_ARM64)
 if (compiler->opts.compDbgEnC)
 {
 // When the EnC option is set we have an exact set of registers that we always save
 // that are also available in future versions.
 availableIntRegs &= ~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED;
 availableIntRegs &= (~RBM_INT_CALLEE_SAVED | RBM_ENC_CALLEE_SAVED).GetIntRegSet();
#if defined(UNIX_AMD64_ABI)
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED;
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED;
#else
 availableFloatRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
 availableDoubleRegs &= ~RBM_FLT_CALLEE_SAVED.GetFloatRegSet();
#endif // UNIX_AMD64_ABI
#if defined(TARGET_XARCH)
 availableMaskRegs &= ~RBM_MSK_CALLEE_SAVED;
#endif // TARGET_XARCH
Expand All@@ -889,8 +891,8 @@ LinearScan::LinearScan(Compiler* theCompiler)
#if defined(TARGET_AMD64)
 if (compiler->canUseEvexEncoding())
 {
 availableFloatRegs |= RBM_HIGHFLOAT;
 availableDoubleRegs |= RBM_HIGHFLOAT;
 availableFloatRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 availableDoubleRegs |= RBM_HIGHFLOAT.GetFloatRegSet();
 }
#endif

Expand DownExpand Up@@ -2813,7 +2815,7 @@ void LinearScan::setFrameType()
 SingleTypeRegSet removeMask = RBM_NONE;
 if (frameType == FT_EBP_FRAME)
 {
 removeMask |= RBM_FPBASE;
 removeMask |= RBM_FPBASE.GetIntRegSet();
 }

 compiler->rpFrameType = frameType;
Expand All@@ -2826,7 +2828,7 @@ void LinearScan::setFrameType()
 compiler->codeGen->regSet.rsMaskResvd |= RBM_OPT_RSVD;
 assert(REG_OPT_RSVD != REG_FP);
 JITDUMP(" Reserved REG_OPT_RSVD (%s) due to large frame\n", getRegName(REG_OPT_RSVD));
 removeMask |= RBM_OPT_RSVD;
 removeMask |= RBM_OPT_RSVD.GetIntRegSet();
 }
#endif // TARGET_ARMARCH || TARGET_RISCV64

Expand DownExpand Up@@ -4039,7 +4041,7 @@ void LinearScan::processKills(RefPosition* killRefPosition)
{
 RefPosition* nextKill = killRefPosition->nextRefPosition;

 regMaskTP killedRegs = killRefPosition->registerAssignment;
 regMaskTP killedRegs = killRefPosition->getKillRegisterAssignment();
 while (killedRegs.IsNonEmpty())
 {
 regNumber killedReg = genFirstRegNumFromMaskAndToggle(killedRegs);
Expand All@@ -4059,9 +4061,9 @@ void LinearScan::processKills(RefPosition* killRefPosition)
 updateNextFixedRef(regRecord, regNextRefPos, nextKill);
 }

 regsBusyUntilKill &= ~killRefPosition->registerAssignment;
 regsBusyUntilKill &= ~killRefPosition->getKillRegisterAssignment();
 INDEBUG(dumpLsraAllocationEvent(LSRA_EVENT_KILL_REGS, nullptr, REG_NA, nullptr, NONE,
 killRefPosition->registerAssignment));
 killRefPosition->getKillRegisterAssignment()));
}

//------------------------------------------------------------------------
Expand DownExpand Up@@ -8811,7 +8813,7 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 {
 // Exclude any doubles for which the odd half isn't in freeRegs,
 // and restrict down to just the even part of the even/odd pair.
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH) >> 1;
 freeRegs &= (freeRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1;
 }
#endif

Expand All@@ -8822,13 +8824,9 @@ regNumber LinearScan::getTempRegForResolution(BasicBlock* fromBlock,
 else
 {
 // Prefer a callee-trashed register if possible to prevent new prolog/epilog saves/restores.
 if ((freeRegs & RBM_CALLEE_TRASH) != 0)
 if ((freeRegs & RBM_CALLEE_TRASH.GetRegSetForType(type)) != 0)
 {
#ifdef TARGET_XARCH
 freeRegs &= RBM_CALLEE_TRASH.GetRegSetForType(type);
#else
 freeRegs &= RBM_CALLEE_TRASH;
#endif
 }

 regNumber tempReg = genRegNumFromMask(genFindLowestBit(freeRegs), type);
Expand DownExpand Up@@ -13595,7 +13593,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::select(Interval*
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand DownExpand Up@@ -13918,7 +13916,7 @@ SingleTypeRegSet LinearScan::RegisterSelection::selectMinimal(
 // clause below creates a mask to do this.
 if (currentInterval->registerType == TYP_DOUBLE)
 {
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH) >> 1);
 candidates &= ~((busyRegs & RBM_ALLDOUBLE_HIGH.GetFloatRegSet()) >> 1);
 }
#endif // TARGET_ARM

Expand Down
Loading