Added tensor support for strings and for getting TF_DataType by anshuman23 · Pull Request #7 · anshuman23/tensorflex

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

anshuman23 · 2018-05-23T11:56:33Z

@josevalim, In this PR I have added some functions that were working perfectly. However I have actually been devoting a lot of time to getting TF_INT64 and TF_FLOAT tensors (especially multi-dimensional ones) to work in Elixir. I will add PRs for those soon, once I get them to work completely.

In this PR, the functionalities added are:

Support for string tensors
Getting to know the datatype of a tensor (Will be useful later on, when Tensor support is varied across many datatypes. I have already added support for all datatypes in this function)

An example for these functions is as follows:

iex(1)>tensor=Tensorflex.string_tensor"string tensors work!"#Reference<0.697833995.47579139.50516>iex(2)>tensor2=Tensorflex.string_tensor123{:error,:non_binary_argument}iex(3)>Tensorflex.tensor_datatypetensor{:ok,:tf_string}

josevalim · 2018-05-26T09:02:34Z

+ memcpy((void *) tensor_resource_alloc, (void *) &tensor, sizeof(TF_Tensor *));
+ ERL_NIF_TERM new_tensor = enif_make_resource(env, tensor_resource_alloc);
+ enif_release_resource(tensor_resource_alloc);
+ return new_tensor;


Since we are returning {:error, ...} in failure cases, it is recommended to return {:ok, tensor} in success cases.

Agreed. In the next PR, I'll add this. I'll extend it to read_graph too

josevalim · 2018-05-26T09:05:33Z

+ TF_Tensor **tensor_resource_alloc = enif_alloc_resource(tensor_resource, sizeof(TF_Tensor *));
+
+ if (!(enif_is_binary(env, argv[0]))) {
+	return enif_make_tuple2(env, enif_make_atom(env,"error"), enif_make_atom(env,"non_binary_argument"));


If the error happens because of bad input, it is preferrable to just raise an error: maybe we can use enif_make_badarg? http://erlang.org/doc/man/erl_nif.html#enif_make_badarg
In a nutshell, we return tuple errors for semantic errors: for example, you couldn't load a file because it is corrupt or it doesn't exist. For language errors, such as invalid syntax or invalid arguments, we should raise instead.

Understood. Will incorporate this in the next PR as well

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Added tensor support for strings and for getting TF_DataType
e3490a1

anshuman23 merged commit 972f53c into masterMay 25, 2018

josevalim reviewed May 26, 2018
View reviewed changes

Conversation

anshuman23 commented May 23, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anshuman23 commented May 23, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anshuman23 commented May 23, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anshuman23 commented May 23, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anshuman23 commented May 23, 2018

Uh oh!

Choose a reason for hiding this comment