Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Initial linux touch implementation - #1706

Closed
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly
Closed

Initial linux touch implementation#1706
ap4ss3rby wants to merge 14 commits into
LizardByte:nightlyfrom
ap4ss3rby:nightly

Conversation

@ap4ss3rby

@ap4ss3rbyap4ss3rby commented Oct 5, 2023

Copy link
Copy Markdown

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

  • Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter
  • Report that we infact have touch capability on linux hosts
  • Probably break absolute mouse mode even more
  • Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
  • Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.
Needs a properly patched Moonlight client for android testing
@CLAassistant

CLAassistant commented Oct 5, 2023

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ReenigneArcher
ReenigneArcher marked this pull request as draft October 5, 2023 22:19
@ap4ss3rbyap4ss3rby changed the title Initial linux touch implementationWIP: Initial linux touch implementationOct 5, 2023
@ap4ss3rbyap4ss3rby changed the title WIP: Initial linux touch implementationInitial linux touch implementationOct 7, 2023
@ap4ss3rby
ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52
@ap4ss3rby

ap4ss3rby commented Oct 7, 2023

Copy link
Copy Markdown
Author

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

get_capabilities() {
return 0;
platform_caps::caps_t caps = 0;
caps |= platform_caps::pen_touch;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement but will require external testing since I don't have anything capable of doing pen events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can test with my devices

libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

Comment threadsrc/platform/common.h Outdated
Comment threadsrc/platform/linux/input.cpp Outdated
Comment threadsrc/platform/linux/input.cpp
@WesleyVanNeck

Copy link
Copy Markdown

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

@WesleyVanNeck

Copy link
Copy Markdown

on kde it works fine do the mouse cursor shows up

@ReenigneArcher

Copy link
Copy Markdown
Member

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ap4ss3rby@CLAassistant@WesleyVanNeck@ReenigneArcher@cgutman