Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
[performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); [performance] Import the large project by snjeza · Pull Request #17 · eclipseguru/bazel-eclipse · GitHub
Skip to content

[performance] Import the large project - #17

Closed
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16
Closed

[performance] Import the large project#17
snjeza wants to merge 1 commit into
eclipseguru:mainfrom
snjeza:issue-16

Conversation

@snjeza

@snjezasnjeza commented Nov 5, 2025

Copy link
Copy Markdown

Fixes#16

Steps to test:

bazel build //...
  • add
directories:
# . # import everything (remove the dot if this is too much)
module1496_public
module0472_public
module0910
module0910_public
module1805_public
module1805
derive_targets_from_directories: true
target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

  • add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
  • import the project in Eclipse without the PR
  • The first time it takes about 25+ minutes to import the project on my machine.
  • call Sync Bazel Projects View
  • The action now takes 20+ minutes.
  • delete all projects
  • import the project in Eclipse with the PR
  • The first time it takes about 20 seconds to import on my machine.
  • call Sync Bazel Projects View
  • The action now takes 10 seconds.

@github-actions

github-actionsBot commented Nov 5, 2025

Copy link
Copy Markdown

Test Results

 42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0 
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0 

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

@snjeza
snjezaforce-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3dCompareNovember 27, 2025 15:53
@guw
guwforce-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953CompareNovember 28, 2025 15:32
@snjeza
snjezaforce-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7CompareDecember 3, 2025 18:16
@snjezasnjeza changed the title Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large[performance] Import the large projectDec 3, 2025
@snjeza

Copy link
Copy Markdown
Author

The PR includes the folowing changes:

  • minimized the number of 'bazel query...'
  • minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
  • minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
  • minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
  • fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
  • replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

@guwguw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

} finally {
try {
refreshResources(resourcesToRefresh, monitor);
//refreshResources(resourcesToRefresh, monitor);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this disabled?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to take a look at #16 , issue 4.


// loads can be potentially expensive; we synchronize on the location
var location = getLocation();
var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

* @return
*/
public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
Supplier<I> supplier);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.


private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

.collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
}

public void put(BazelLabel label, IProject project) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A public put is problematic. What is the reason for this?

// during synchronization resource changes may occur; however, they are triggered by the synchronization activities
// therefore we suspend cache invalidation of the model due to resource changes
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bug fix?

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I have created #44

public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
throws CoreException {
// bazel query 'kind(rule, deps(//foo:all + //bar:all))"'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

@snjezasnjezaDec 8, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created #43

@snjeza

Copy link
Copy Markdown
Author

@snjezasnjeza closed this Dec 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large

2 participants

@snjeza@guw