Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Repository files navigation

A Really Dumb Xcode 11 Build Issue

Stick with Objective-C, and you'll get the fastest builds, EVER!

Yeah, maybe not.

When I build my app, Xcode 11 appears to stall out completely (dropping from full core usage down to 10-20% CPU) during the Objective-C build phases.

In contrast, my massive Swift targets will happily peg my machine with all the cores/threads getting utilized. Swift doesn't get tripped up by this bug at all.

Xcode CPU Usage

Background

This project demonstrates a horrible bug in the Xcode 11 build system (which may be happening somewhere as deep as LLVM/clang).

The bug manifested for me like this:

  1. Starting with a clean DerivedData folder, I can build my app, Capo in about 60 seconds, flat.
  2. Once I clean the build folder using command-shift-K, all subsequent builds will take anywhere from 180s to 500s (in the very worst case, which I've not seen for a while now.)

After some back & forth with Apple's engineers, I learned to turn on clang's -Rmodule-build flag to see what's going on behind the scenes. It turns out that—for every Objective-C file that is built—framework modules are being re-built unnecessarily.

Instructions

First, clone this repository:

git clone https://github.com/liscio/SubFrameworks

Command-Line Version (Preferred/Faster)

Run the included checkRebuilds.sh script at the command-line.

You will see pre-filtered output like this:

%%% Building Using Clean DerivedData %%%
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: building module 'Foundation' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/Foundation-2PSPYZLNW14KI.pcm' [-Rmodule-build]
SubFrameworks/SubFrameworks/SubB.h:9:9: remark: finished building module 'Foundation' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
%%% Clean re-build %%%
SubFrameworks/MainFramework/MainF.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainF.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainG.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainC.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainD.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainE.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainA.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: building module 'SubFrameworks' as 'SubFrameworks/testDerivedData/ModuleCache.noindex/33MFFE8RLECJA/SubFrameworks-3U5BOV1R8307A.pcm' [-Rmodule-build]
SubFrameworks/MainFramework/MainB.h:10:9: remark: finished building module 'SubFrameworks' [-Rmodule-build]

What am I looking at here?

The script starts off by building the MainFramework target with a fresh, local testDerivedData folder.

The first block of output demonstrates that Xcode correctly builds modules for Foundation, and SubFrameworks only once. That is what the output should always look like.

The second block of output is the result of asking xcodebuild to perform a clean build action. You'll notice that Foundation is not re-built, however SubFrameworks is built *for every single .m file in the MainFramework target. I would have expected to see it only once since we are explicitly triggering its re-build.

Imagine that SubFramework contains your application's (extensive) core logic, and hundreds of source files. Now imagine that a good number of the source files in your main application #import <SubFramework/SubFramework.h>. For each of those source files, the SubFramework module is re-compiled again and again.

Crazy, right?!

Using Xcode

You can also reproduce the same result in Xcode 11.x, which is where I spend most of my time working.

Open the Xcode project, and do the following:

  1. Choose the MainFramework target
  2. Build the product. You'll see warnings in the output that show which modules are being built.
  3. Press Command-Shift-K to clear the build folder
  4. Build the product again. You'll now see warnings indicating that the SubFramework module is re-built for each source file.

Does the Xcode Version Matter?

Yes. This bug does not appear to affect Xcode 10.x. All of my installed Xcode 11.x releases (11.0 - 11.3 beta) are affected.

Hey, Apple Folks!

For anybody that cares to look further into this, this specific issue is tracked by FB7046188.

What's especially frustrating is that I've been experiencing terrible build performance since April 2018. See also FB5715747, and FB5642600.

Affected Systems

I have only been able to test this on my iMac Pro, and a 2016 MacBook Pro. Both machines have experienced this bug on Mojave and Catalina.

About

A really, really dumb Xcode/LLVM/clang bug…

Resources

Stars

26 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages