Add better support for GlobalId and active job by yuki24 · Pull Request #150 · codegram/hyperclient

yuki24 · 2019-08-25T15:36:22Z

This PR adds better support for ActiveJob and GlobalId and will make it dramatically. This is still a WIP, but Artsy will start using this branch shortly.

Before

yuki=Gravity.client.user_detail(id: "yuki-id")UsersMailer.reset_password(user_detail: yuki,reset_password_token: 'xyz').deliver_later# => ActiveJob::SerializationError: Unsupported argument type: Hyperclient::Link

After

yuki=Gravity.client.user_detail(id: "yuki-id")UsersMailer.reset_password(user_detail: yuki,reset_password_token: 'xyz').deliver_later# => Enqueued ActionMailer::DeliveryJob (Job ID: ...)

Todo

Add tests for the #to_global_id method
Add tests for the #to_signed_global_id method
Implement the same interface in Hyperclient::Link
Implement the same interface in Hyperclient::LinkCollection
Implement the same interface in Hyperclient::Resource
Implement the same interface in Hyperclient::ResourceCollection

dblock

I worry that we're bringing in the kitchensink of Rails. Would it make more sense to make a separate library hyperclient-globalid or hyperclient-rails instead

dblock · 2019-08-25T21:35:54Z

 # Public: Hyperclient namespace.
 #
 module Hyperclient
+ URL_TO_ENDPOINT_MAPPING = { }


This isn't a constant, shouldn't be CAPS.

This reference seems constant, as it's the object itself that will be mutated, right?

yuki24 · 2019-08-25T23:26:48Z

Hmm I understand the concern but smaller gems are just a burden to maintain. But I'm not strongly against that idea.

ivoanjo

Left a few notes!

Since no new dependency is needed to support this, may I suggest just keeping all of the changes in a separate folder (which would be basically the folder that would be moved over to a hypothetical hyperclient-rails), and only adding the conditional load on hyperclient.rb?

That way I think we can get the best of both worlds: we keep the rails changes clearly separated, but we don't need the extra dependency and since we test both hyperclient and the rails changes together, we also get the advantage that we don't accidentally break it with some other future change.

ivoanjo · 2019-08-26T06:49:21Z

+ }
+ end
+ end
+end


Minor: Don't forget to configure your editor to add a newline at the end of the file :)

ivoanjo · 2019-08-26T06:54:58Z

+ ::GlobalID::Locator.use app_name, ::Hyperclient::GlobalId::Locator.new
+ end
+
+ def to_global_id(options = {})


Minor: I suggest using the more modern **options e.g. to_global_id(**options), as it also ensures that all keys are symbols, thus avoiding mistakes when using strings instead.

ivoanjo · 2019-08-26T07:28:48Z

 # Returns a Hyperclient::EntryPoint
 def self.new(url, &block)
- Hyperclient::EntryPoint.new(url, &block)
+ URL_TO_ENDPOINT_MAPPING[url] = Hyperclient::EntryPoint.new(url, &block)


Hello there! I may be missing a bit of context on how this works, but I have a few notes:
This creates a memory leak -- every time we create a new Hyperclient instance we add it to this map, and never remove it. Have you considered changing this so it won't grow forever?
This is problematic if you're running with a Ruby with support for parallelism, as many threads can be competing to write to the hash. Since everyone's just writing the intuition may be that there's no problem, and indeed on MRI Ruby since a Hash is implemented in C this has no problem, but on other Rubies this can be seen. Here's my quick experiment:
putsRUBY_DESCRIPTIONPER_THREAD_LIMIT=1000000THREADS=8THE_HASH={}(1..THREADS).to_a.mapdo |thread_id| Thread.newdoPER_THREAD_LIMIT.timesdoTHE_HASH[rand(PER_THREAD_LIMIT)]=thread_idendputs"Thread #{thread_id} done!"endend.map(&:join)puts"All done!"
Running this on TruffleRuby:
truffleruby19.1.1, likeruby2.6.2, GraalVMCENative [x86_64-linux] Thread1done! threads.rb:11:in `[]=': <no message> (NullPointerException) (RuntimeError) fromorg.truffleruby.core.hash.PackedArrayStrategy.getHashed(PackedArrayStrategy.java:41) fromorg.truffleruby.core.hash.SetNode.setPackedArray(SetNode.java:80) fromorg.truffleruby.core.hash.SetNodeGen.executeSet(SetNodeGen.java:37) fromorg.truffleruby.core.hash.HashNodes$SetIndexNode.set(HashNodes.java:239) fromorg.truffleruby.core.hash.HashNodesFactory$SetIndexNodeFactory$SetIndexNodeGen.execute(HashNodesFactory.java:632) fromorg.truffleruby.language.control.SequenceNode.execute(SequenceNode.java:34) fromorg.truffleruby.language.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:51) fromorg.truffleruby.language.RubyRootNode.execute(RubyRootNode.java:54) fromorg.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callProxy(OptimizedCallTarget.java:328) fromorg.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callRoot(OptimizedCallTarget.java:318) Translatedtointernalerrorfromthreads.rb:11:in `block (3levels) in <main>' fromthreads.rb:10:in `times' fromthreads.rb:10:in `block (2levels) in <main>'
Running this on JRuby:
jruby9.2.8.0 (2.5.3) 2019-08-12a1ac7ffOpenJDK64-BitServerVM11.0.3+7-LTSon11.0.3+7-LTS +jit [linux-x86_64] warning: thread"Ruby-0-Thread-3: threads.rb:1"terminatedwithexception (report_on_exceptionistrue): java.lang.ArrayIndexOutOfBoundsException: Index18581outofboundsforlength16427atorg.jruby.dist/org.jruby.RubyHash.internalPutNoResize(RubyHash.java:561) atorg.jruby.dist/org.jruby.RubyHash.internalPut(RubyHash.java:535) atorg.jruby.dist/org.jruby.RubyHash.internalPut(RubyHash.java:525) atorg.jruby.dist/org.jruby.RubyHash.fastASetCheckString(RubyHash.java:1020) atorg.jruby.dist/org.jruby.RubyHash.op_aset(RubyHash.java:1055) atorg.jruby.dist/org.jruby.RubyHash$INVOKER$i$2$0$op_aset.call(RubyHash$INVOKER$i$2$0$op_aset.gen) atorg.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203) atthreads.invokeOther1:\=\{\}=(threads.rb:11) atthreads.RUBY$block$\=threads\,rb$2(threads.rb:11) atorg.jruby.dist/org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146) atorg.jruby.dist/org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:85) atorg.jruby.dist/org.jruby.runtime.Block.yieldSpecific(Block.java:139) atorg.jruby.dist/org.jruby.RubyFixnum.times(RubyFixnum.java:279) atorg.jruby.dist/org.jruby.RubyInteger$INVOKER$i$0$0$times.call(RubyInteger$INVOKER$i$0$0$times.gen) atorg.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:151) atorg.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:160) atthreads.invokeOther3:times(threads.rb:10) atthreads.RUBY$block$\=threads\,rb$1(threads.rb:10) atorg.jruby.dist/org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136) atorg.jruby.dist/org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77) atorg.jruby.dist/org.jruby.runtime.Block.call(Block.java:129) atorg.jruby.dist/org.jruby.RubyProc.call(RubyProc.java:295) atorg.jruby.dist/org.jruby.RubyProc.call(RubyProc.java:274) atorg.jruby.dist/org.jruby.RubyProc.call(RubyProc.java:270) atorg.jruby.dist/org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105) atjava.base/java.lang.Thread.run(Thread.java:834) Thread5done! Thread8done! Thread4done! Thread6done! Thread2done! Thread1done! UnhandledJavaexception: java.lang.ArrayIndexOutOfBoundsException: Index18581outofboundsforlength16427java.lang.ArrayIndexOutOfBoundsException: Index18581outofboundsforlength16427internalPutNoResizeatorg/jruby/RubyHash.java:561internalPutatorg/jruby/RubyHash.java:535internalPutatorg/jruby/RubyHash.java:525fastASetCheckStringatorg/jruby/RubyHash.java:1020op_asetatorg/jruby/RubyHash.java:1055callatorg/jruby/RubyHash$INVOKER$i$2$0$op_aset.gen:-1callatorg/jruby/runtime/callsite/CachingCallSite.java:203invokeOther1:\=\{\}= atthreads.rb:11threads.rbatthreads.rb:11yieldDirectatorg/jruby/runtime/CompiledIRBlockBody.java:146yieldSpecificatorg/jruby/runtime/IRBlockBody.java:85yieldSpecificatorg/jruby/runtime/Block.java:139timesatorg/jruby/RubyFixnum.java:279callatorg/jruby/RubyInteger$INVOKER$i$0$0$times.gen:-1callatorg/jruby/runtime/callsite/CachingCallSite.java:151callIteratorg/jruby/runtime/callsite/CachingCallSite.java:160invokeOther3:timesatthreads.rb:10threads.rbatthreads.rb:10callDirectatorg/jruby/runtime/CompiledIRBlockBody.java:136callatorg/jruby/runtime/IRBlockBody.java:77callatorg/jruby/runtime/Block.java:129callatorg/jruby/RubyProc.java:295callatorg/jruby/RubyProc.java:274callatorg/jruby/RubyProc.java:270runatorg/jruby/internal/runtime/RubyRunnable.java:105runatjava/lang/Thread.java:834
This won't happen often, of course -- I needed to create a few threads and a lot of items, but it's the kind of thing that can bite you at the worst possible time, e.g. when doing a deployment with high-traffic with a threaded webserver.
My suggestion for this one would be to consider using a Concurrent::Map from the concurrent-ruby gem, or using a lock to protect the map while writing.

ivoanjo · 2019-08-26T07:29:55Z

 # Public: Hyperclient namespace.
 #
 module Hyperclient
+ URL_TO_ENDPOINT_MAPPING = { }


This reference seems constant, as it's the object itself that will be mutated, right?

dblock · 2019-08-26T11:37:07Z

Hmm I understand the concern but smaller gems are just a burden to maintain. But I'm not strongly against that idea.

I think in the case of support for Rails it makes more sense to split. It's a common pattern. Rails keeps adding kitchen-sink type things and this makes things harder to maintain. That said I don't have that strong of feelings about it, I just thought I'd bring it up.

ivoanjo · 2019-09-10T22:17:36Z

Hey @yuki24 I hope I didn't scare you with my comments, I'm definitely willing to help out with solving those :)

yuki24 force-pushed the globalid-and-active-job branch 2 times, most recently from 9e09bbb to fc9af92Compare August 25, 2019 15:45

Add basic support for GlobalId
359df52

yuki24 force-pushed the globalid-and-active-job branch from fc9af92 to 359df52Compare August 25, 2019 15:50

dblock reviewed Aug 25, 2019
View reviewed changes

ivoanjo suggested changes Aug 26, 2019
View reviewed changes

Navigation Menu

Uh oh!

Add better support for GlobalId and active job - #150

Add better support for GlobalId and active job#150
yuki24 wants to merge 1 commit into
codegram:masterfrom
yuki24:globalid-and-active-job

yuki24 commented Aug 25, 2019

Uh oh!

dblock left a comment

Uh oh!

dblockAug 25, 2019

Uh oh!

ivoanjoAug 26, 2019

Uh oh!

yuki24 commented Aug 25, 2019

Uh oh!

ivoanjo left a comment

Uh oh!

ivoanjoAug 26, 2019

Uh oh!

ivoanjoAug 26, 2019

Uh oh!

ivoanjoAug 26, 2019•
edited
Loading

Uh oh!

ivoanjoAug 26, 2019

Uh oh!

dblock commented Aug 26, 2019

Uh oh!

ivoanjo commented Sep 10, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

+ }
+ end
+ end
+end No newline at end of file

Uh oh!

Conversation

yuki24 commented Aug 25, 2019

Before

After

Todo

Uh oh!

dblock left a comment

Choose a reason for hiding this comment

Uh oh!

dblockAug 25, 2019

Choose a reason for hiding this comment

Uh oh!

ivoanjoAug 26, 2019

Choose a reason for hiding this comment

Uh oh!

yuki24 commented Aug 25, 2019

Uh oh!

ivoanjo left a comment

Choose a reason for hiding this comment

Uh oh!

ivoanjoAug 26, 2019

Choose a reason for hiding this comment

Uh oh!

ivoanjoAug 26, 2019

Choose a reason for hiding this comment

Uh oh!

ivoanjoAug 26, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ivoanjoAug 26, 2019

Choose a reason for hiding this comment

Uh oh!

dblock commented Aug 26, 2019

Uh oh!

ivoanjo commented Sep 10, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ivoanjoAug 26, 2019•
edited
Loading