JsObject API

[iskiselev]

I suggest to introduce JsObject API, that will allow easier interop with JS, then using Verbutium without usage of dynamic. Suggested API:

public abstract class JsObject
{
    public abstract JsObject this[string name] { get; set; }

    public abstract JsObject Get(string name);
    public abstract T Get<T>(string name);
    public abstract void Set(string name, object value);

    public abstract JsObject Call(string name, params object[] args);
    public abstract T Call<T>(string name, params object[] args);
    public abstract JsObject Create(string name, params object[] args);

    public abstract T Cast<T>();
}

public abstract class JsFunction : JsObject
{
    public abstract JsObject Invoke(params object[] args);
    public abstract T Invoke<T>(params object[] args);
    public abstract JsObject InvokeCreate(params object[] args);
}

Additionally we can overload some operators in JsObject. Initial implementation could be done with JsReplace attribute, but for proper implementation we should implement it inside JSIL transfromations logic - as we should detect creating new array in params calls and replace it with simple method call. Also we may discuss, if JsFunction should catch original this context or not.

EDITED: Suggested API changed based on @kg feedback

[iskiselev]

This issue is connected with #531.

[kg]

JsFunction.Call should be named JsFunction.Invoke. Otherwise this is ambiguous:

JsObject o; JsFunction f;

o.Call("x", 1);
o.Call("y");

f.Call();
f.Call(1);

f.Call("y"); // ????????

[kg]

JsObject.Cast's intent is ambiguous; it is unclear whether it is a conversion or an 'I promise this value is of type T' operation. For JsObject I would instead do something like:

public abstract T AssumeType<T>();
public abstract T As<T> () where T : class;
public abstract T Cast<T> ();
public abstract bool Is<T> ();

Sadly we can't support as/is/(T) on JsObject since the compiler will yell that the conversions are impossible :-(

[kg]

Also worthwhile to add:

public abstract JsObject this[int index] { get; set; }

[iskiselev]

Also it may be good idea add to JsObject for iterating over keys in object.:

public abstract IEnumerable<string> In()

[kg]

Ah, yes. Though I would probably name that one Keys. Good point either way.

[iskiselev]

Could you define, how should Cast<T> work for JsObject per you example? My initial thought was only introducing it with AssumeType<T> semantics.

[kg]

It would generate a checked static cast at runtime - T.$Cast(X). Same for As and Is.

[Bablakeluke]

Hey guys, I've started on the above, specifically on the libraries rather than the compiler side; my goal at this point is to essentially produce the set of core lib DLLs based on the Mono ones using the above API. Quick overview:

• This will happen over in WebIL to prevent polluting you guys too much :)
• Find all extern methods (and similar; anything which relies on the CLR in short) in the Mono DLLs
• Generate a set of C# files (93 of those) containing these methods (564 of them)
• Implement the method bodies using an API like this one, basing the code off the existing JSIL library implementations
• Compile those C# files
• Merge back with the original DLLs, replacing the extern methods with the JS implemented ones

The end result would be a set of "standalone" DLLs which could then be passed through the JSIL compiler.

[iskiselev]

@Bablakeluke, I'm not sure that fully understand your goals. Which Mono libraries you want to translate? mscorlib/System.Core/System? What do you want to achieve with it?
I have successful experience of translating that libraries (really it works with MS .Net a little bit better for now). Main problem right now is conflicts of types from that assemblies and types that were already defined in JSIL Bootstap JS files. We can't live without bootstrap, but it is not easy task mix them with translated parts (some work for it is inside #857).
External implementation for most classes in BCL library is not worth problem - as (1) this class could not be used a lot or (2) already has implementation in JSIL Bootstrap.

Probably I just not fully understand your desires. If you describe them more (probably with examples) I may suggest you some better options :)

[Bablakeluke]

Sure, I'll try and describe it a little better: In short, Reflection.Emit support. This single feature requires a completely different direction and, in my case at least (a moddable game currently in the Unity web player with its own custom scripting languages), is vital to have. Ideally, I want to leverage as much existing code as possible. That's where the Mono libraries come in - All of them! The point of view is that the runtime must be able to load a bytecode translated from emit calls anyway, so it might as well load the libraries in this way too. That allows it to keep the compactness of streaming IL vs streaming potentially enormous amounts of pre-translated .NET IL in Javascript.

So, a few things about this approach. Keeping in line with reusing as much code as possible, the actual bytecode and files being streamed in would actually be WebAssembly but with a small GC extension (which is coming in the distant future of WebAssembly anyway, and the WebIL project will essentially be making proposals on how it should operate). This avoids needing to implement a complete .NET runtime in Javascript and it'll be accelerated by all major browsers shortly too. A quick summary at this point, then, is like this:

Mono DLLs -> WebAssembly files (+GC calls) -> stream to browser -> Load on a runtime

Next, ideally, that runtime is as small as it can possibly be. Totally free from any particular languages/ frameworks like .NET, if possible - allowing it to leverage the existing JS engines. That means all the bootstrap-like functionality needs to be contained within those special WebAssembly files, which are being derived from ordinary DLLs. This is where JSObject comes in.

Getting JS calls into the Mono DLLs

The Mono DLLs of course contain a bunch of extern methods - the stubs which point at the functionality implemented in the bootstrap/ runtime. However, I need to implement them inside the library itself - get JS calls into the body of the methods. That process works like this:

A Mono DLL -> Get all the extern/ InternalCall methods -> Generate a bunch of C# files with those methods in them -> Implement the methods in C# with JSObject calls -> Compile them into a DLL -> Merge this DLL with the original Mono one, overwriting the original extern call. From there, the resulting DLL is the one which then goes ahead and gets converted into a WebAssembly file.

(It's worth pointing out here that the Mono DLL is used rather than the sources as it provides a nice wall between Mono/WebIL)

So, time for a bit of an example! At the moment, the generation/ compilation/ merge of the C# files is working, and Socket seems like a nice place to start with an example as feature wise, it's about as awkward as they get (ironically, inside a web browser). In this case, the implementation would be the original protocol over websocket or WebRTC, and it's just going to be a very basic overview for now. Looking in the C# files generated from the Mono DLL, there's this method:

private static void Connect_internal_real(IntPtr sock, SocketAddress sa, out int error){

throw new NotSupportedException();

}

First thing to note is JSObject and IntPtr are essentially the same thing. So, the body would look something like this:

private static void Connect_internal_real(IntPtr sock, SocketAddress sa, out int error){

// Give the compiler some context (this line isn't actually here in the final output - it's purely for the compile pipeline):
JSObject jsObject=new JSObject(sock);

// Get the port (3rd and 4th bytes are port, little endian, everything after is IP):
int port=sa.Item[2] | (sa.Item[3]<<8);

// Get the IP (v4 only here for now):
string ip=sa.Item[4]+"."+sa.Item[5]+"."+sa.Item[6]+"."+sa.Item[7];

// Create a websocket:
// In vanilla JS, this is equiv to new websocket("ws://"+ip+":"+port+"/");
JSObject webSocket=JS.Import("websocket").New("ws://"+ip+":"+port+"/");

// Other examples of the API:
// webSocket.url
// string url=webSocket["url"] as string;

// webSocket.close();
// webSocket.Invoke("close",n,params,would,go,here);

// Overwrite the thing at the original pointer:
jsObject.Set(webSocket);

}

The code above is of course rough, so only take it as an overview of the API. Importantly, it's all (probably!) valid C#, aka it will compile into a DLL. This method inside that DLL can then be used to overwrite the original Mono extern method with the same namespace and method name, creating a method which is now only dependent on the JSObject API. In short, the conversion to WebAssembly implements the relatively tiny API, resulting in a "standalone" WebAssembly file, without the need for built in methods. From there, the compiler can do all the hard work, stripping IL methods that are known to be not in use, or the pipeline could be used for other languages like Java too.

[iskiselev]

At least for initial implementation, I'd like to keep JS interop API as simple, as possible.
My proposal for first cut:

    public abstract class JsObject
    {
        public abstract JsObject this[string name]
        {
            [JSReplacement("$this[$name]")]
            get;
            [JSReplacement("$this[$name] = $value")]
            set;
        }
    }

    public abstract class JsFunction
    {
        [JSReplacement("$this()")]
        public abstract JsObject Invoke();
        [JSReplacement("$this($arg0)")]
        public abstract JsObject Invoke<TArg0>(TArg0 arg0);
        [JSReplacement("$this($arg0, $arg1)")]
        public abstract JsObject Invoke<TArg0, TArg1>(TArg0 arg0, TArg1 arg1);
        [JSReplacement("$this($arg0, $arg1, $arg2)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2>(TArg0 arg0, TArg1 arg1, TArg2 arg2);
        [JSReplacement("$this($arg0, $arg1, $arg2, $arg3)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2, TArg3>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3);
        [JSReplacement("$this($arg0, $arg1, $arg2, $arg3, $arg4)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2, TArg3, TArg4>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4);
        [JSReplacement("$this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5);
        [JSReplacement("$this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5, $arg6)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5, TArg6>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5, TArg6 arg6);
        [JSReplacement("$this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5, $arg6, $arg7)")]
        public abstract JsObject Invoke<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5, TArg6, TArg7>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5, TArg6 arg6, TArg7 arg7);

        [JSReplacement("new $this()")]
        public abstract JsObject InvokeNew();
        [JSReplacement("new $this($arg0)")]
        public abstract JsObject InvokeNew<TArg0>(TArg0 arg0);
        [JSReplacement("new $this($arg0, $arg1)")]
        public abstract JsObject InvokeNew<TArg0, TArg1>(TArg0 arg0, TArg1 arg1);
        [JSReplacement("new $this($arg0, $arg1, $arg2)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2>(TArg0 arg0, TArg1 arg1, TArg2 arg2);
        [JSReplacement("new $this($arg0, $arg1, $arg2, $arg3)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2, TArg3>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3);
        [JSReplacement("new $this($arg0, $arg1, $arg2, $arg3, $arg4)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2, TArg3, TArg4>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4);
        [JSReplacement("new $this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5);
        [JSReplacement("new $this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5, $arg6)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5, TArg6>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5, TArg6 arg6);
        [JSReplacement("new $this($arg0, $arg1, $arg2, $arg3, $arg4, $arg5, $arg6, $arg7)")]
        public abstract JsObject InvokeNew<TArg0, TArg1, TArg2, TArg3, TArg4, TArg5, TArg6, TArg7>(TArg0 arg0, TArg1 arg1, TArg2 arg2, TArg3 arg3, TArg4 arg4, TArg5 arg5, TArg6 arg6, TArg7 arg7);
    }

    public static class JsObjectHelpers
    {
        [JSReplacement("$obj")]
        public static T AssumeType<T>(this JsObject obj)
        {
            throw new NotImplementedException();
        }

        [JSReplacement("$T.$$As($obj)")]
        public static T As<T>(this JsObject obj) where T : class
        {
            throw new NotImplementedException();
        }

        [JSReplacement("$T.$$Cast($obj)")]
        public static T Cast<T>(this JsObject obj)
        {
            throw new NotImplementedException();
        }

        [JSReplacement("$T.$$Is($obj)")]
        public static bool Is<T>(this JsObject obj)
        {
            throw new NotImplementedException();
        }
    }

Verbatium.Expression, JSGlobal, JSLocal will return JsObject.
My next step will be intorducing JS implementation for all interop classes. With it Reflection and dynamic will work normally.

Do we really need any other methods for JsObject/JsFunction?

And now small test example:

          var obj = Verbatim.Expression("{prop: function() {this.inner=function() { this.func = function() {return 10.1 }}}}");

          var num = obj
                ["prop"].AssumeType<JsFunction>().InvokeNew()
                ["inner"].AssumeType<JsFunction>().InvokeNew()
                ["func"].AssumeType<JsFunction>().Invoke().Cast<int>() + 10;
          Console.WriteLine(num);