[Need help]how to call Request.prototype.read outside callback

[horihiro]

I found a commit message 'allow read to be called outside of callback' for modules/network/socket/lwip/modSocket.c.

It seems that the message means Request.prototype.read can be called outside of its callback.

But I got the following error message when I tried.
/PATH/TO/main.js (19) # Break: (host): cannot coerce undefined to object!

Could you please provide me an example for calling Request.prototype.read outside of request's callback if it can be?

The code and manifest I tried is following:

main.js

import {Request} from "http"
import SecureSocket from "securesocket";
import Timer from 'timer';

const request = new Request({
  host: 'translate.google.com',
  port: 443,
  method: 'GET',
  Socket: SecureSocket,
  secure: {protocolVersion: 0x303}

});
request.callback = function(message, value, etc) {
  trace(`${message}:${value||''}\n`);
};

Timer.repeat((id) => {
  if (request.state < 4) return;
  // call Request.prototype.read outsize of a callback of the request
  trace(`${request.read(String)}\n`);
}, 1);

manifest.json

{
  "include": [
    "$(MODDABLE)/examples/manifest_base.json",
    "$(MODDABLE)/examples/manifest_net.json",
    "$(MODDABLE)/modules/crypt/tls.json",
  ],
  "resources": {
    "*": [
      "$(MODULES)/crypt/data/ca23",
    ]
  },
  "modules": {
    "*": [
      "./main",
      "$(MODULES)/network/http/*",
    ]
  },
  "preload": [
    "http",
  ],
}

Best Regards,

[phoddie]

First, you have misunderstood the commit message, "allow read to be called outside of callback", you quote above. That is on socket. Your example does not use socket directly, but uses the http client with a SecureSocket. That puts you two layers away from socket. The commit message does not say anything about higher layers of the stack.

Second, your code is using internal state of the http client, specifically request.state < 4. That's unreliable and unnecessary.

Third, as a rule code should read network data as soon as possible - e.g. in the callback. These microcontrollers have limited memory. The longer you wait to read the data, the more likely you are running with full network buffers, which means that the data rate will be reduced.

All that said, it is a trivial change to the http client do what you want for non-secure connections...

Change the start of the read function in http.js:

	read(type, limit) {
		if (undefined === type)
			return this.socket.read();

With that in place, the example below works on ESP8266 and ESP32. It is a modified version of the httpgetstreaming example.

import {Request} from "http";
import Timer from "timer";

let host = "blog.moddable.com";
let path = "/blog/wp-content/uploads/2019/04/rotating-cards-compressed.gif";
let request = new Request({host, path});
let readable = false;
request.callback = function(message, value, etc) {
	if (Request.responseFragment === message)
		readable = true;

	if (Request.responseComplete == message)
		readable = false;
}

Timer.repeat(() => {
	if (readable && request.read())
		request.read(ArrayBuffer);
}, 10);

Note: Not all host platforms support reading from outside the network callback. The commit you reference above is for lwip sockets, used on ESP8266 and ESP32.

All that said, the SecureSocket implementation expects the callback to consume all the data available to read. Changing that is not trivial.

Further, the example above has a good chance of missing the last few buffers received before responseComplete. When the connection is complete (e.g. on return from responseComplete), any unread buffers are freed.

Therefore, it might be best to read the data in the callback. ;)

[horihiro]

Thanks!!