Add Prism.node_for(Method|UnboundMethod|Proc) to get a Prism node for a callable

[eregon]

• See https://bugs.ruby-lang.org/issues/21005
  and https://bugs.ruby-lang.org/issues/20999

[tompng]

Looks like Proc generated by for in syntax is missing

[eregon]

The 2.7 failure is quite annoying: https://github.com/ruby/prism/actions/runs/20211767571/job/58018650394?pr=3808
It can't parse def inline_method = 42. We can't wrap that in eval otherwise Prism.node_for won't work on it.
I suppose we could use a separate file, but it feels messy.
I tried to repro locally but the main Gemfile doesn't even work in 2.7, so it seems difficult to repro & test locally.
I wish we could just drop 2.7, it has this partial kwargs separation which makes it pretty messy in general.

[Earlopain]

I wish we could just drop 2.7, it has this partial kwargs separation which makes it pretty messy in general.

This is something we should probably talk about sometime in the future. It was added for the convenience of rubocop as far as I'm aware and I appreciate that this happened. But I do not want to keep these long EOL versions supported forever.

For now I would just put this in a separate file like you said. You shoudl be able to use the version-specific gemfile with BUNDLE_GEMFILE=gemfiles/2.7/Gemfile, that works for me at least.

[eregon]

Looks like Proc generated by for in syntax is missing

@tompng Do you mean:

$ bin/parse -e 'for e in foo; body; end'
@ ProgramNode (location: (1,0)-(1,23))
├── flags: ∅
├── locals: [:e]
└── statements:
@ StatementsNode (location: (1,0)-(1,23))
├── flags: ∅
└── body: (length: 1)
└── @ ForNode (location: (1,0)-(1,23))
├── flags: newline
├── index:
│ @ LocalVariableTargetNode (location: (1,4)-(1,5))
│ ├── flags: ∅
│ ├── name: :e
│ └── depth: 0
├── collection:
│ @ CallNode (location: (1,9)-(1,12))
│ ├── flags: variable_call, ignore_visibility
│ ├── receiver: ∅
│ ├── call_operator_loc: ∅
│ ├── name: :foo
│ ├── message_loc: (1,9)-(1,12) = "foo"
│ ├── opening_loc: ∅
│ ├── arguments: ∅
│ ├── closing_loc: ∅
│ ├── equal_loc: ∅
│ └── block: ∅
├── statements:
│ @ StatementsNode (location: (1,14)-(1,18))
│ ├── flags: ∅
│ └── body: (length: 1)
│ └── @ CallNode (location: (1,14)-(1,18))
│ ├── flags: newline, variable_call, ignore_visibility
│ ├── receiver: ∅
│ ├── call_operator_loc: ∅
│ ├── name: :body
│ ├── message_loc: (1,14)-(1,18) = "body"
│ ├── opening_loc: ∅
│ ├── arguments: ∅
│ ├── closing_loc: ∅
│ ├── equal_loc: ∅
│ └── block: ∅
├── for_keyword_loc: (1,0)-(1,3) = "for"
├── in_keyword_loc: (1,6)-(1,8) = "in"
├── do_keyword_loc: ∅
└── end_keyword_loc: (1,20)-(1,23) = "end"

does not have a BlockNode?
I suppose we could return the ForNode#statements or probably better the ForNode itself.

[eregon]

or probably better the ForNode itself.

Implemented.
@tompng Thank you for the reminder, I didn't think of the block given to for.

[eregon]

Looking at things like node.compact_child_nodes.each maybe we should have node.each_child_node? It would avoid the Array allocation and since there is no point to yield nil there is no need to name it node.each_compact_child_node.
Definitely something for another PR, but @kddnewton WDYT?

[eregon]

I found that {Method,UnboundMethod,Proc}#source_location returns columns in bytes and not in characters.
I think that's a bug and filed https://bugs.ruby-lang.org/issues/21783 but nevertheless it's not a problem for this PR.

EDIT: It also means line_and_character_column_to_byte_offset isn't used in this PR anymore but I think it's a good addition anyway, and it will become used again when/if that CRuby bug is fixed.

[eregon]

rfind would be nice here but of course it's only RUby 4+.
I guess we could do a polyfill but for this case I doubt it matters, most of the time is not spent here but in creating the Ruby objects for the AST, there are typically only a few nodes returned by tunnel.

[eregon]

Re the test-all failure, I'll fix it, makes it clear test-unit assert_raise is quite problematic: test-unit/test-unit#347

[Earlopain]

For reference, this won't be available (at least in Ruby 4.0) unfortunately https://bugs.ruby-lang.org/issues/21783#note-8

[kddnewton]

@eregon we've marked this as blocked until and unless the source location thing gets resolved. Another option is requesting an official node_id API. Is that something you would consider implementing in TruffleRuby?

[eregon]

ruby/ruby#15580 is merged so this is unblocked now :)
Though there is no ruby-dev-builder build with it yet, but that should be done soon.

[eregon]

The CI was green before updating to latest CRuby 4.1: https://github.com/ruby/prism/actions/runs/20605030946/attempts/2?pr=3808
And now it's red due to issues unrelated to this PR:
https://github.com/ruby/prism/actions/runs/20605030946/job/59206136960?pr=3808

sexp_processor-4.17.4 requires ruby version < 4, >= 2.6, which is incompatible
with the current version, 4.1.0.dev

Upgrading to sexp_processor-4.17.5 should fix that.

[eregon]

Green now

[kddnewton]

The code looks largely fine, but it's not clear to me that this feature is actually sticking around, based on the discussions on the issue tracker. Let's wait until that's resolved.

[eregon]

Which discussion are you thinking to?
I guess https://bugs.ruby-lang.org/issues/6012#note-40 ?
Matz explicitly approved adding the functionality back in ruby/ruby#15580 (comment)

[eregon]

FWIW I do plan to comment on https://bugs.ruby-lang.org/issues/21795 (sounds great to me).