[random] support random fill

[FrozenGene]

This pr support us to allocate non empty values of nd array, which could solve the issue of autotvm measurement is not correct (see #5200). This is one standalone part of Ansor (#5883).

@kevinthesun@merrymercy@minminsun@jcf94

[tqchen]

see comments, we don't need to expose the(non_empty) function as a primitive function, given that it is not a numpy function, and is only used in AutoTVM, instead we can achieve the same goal by

x=nd.empty(..)
random_fill=get_packed_func("contrib.random.random_fill") random_fill(x)

Notably, the above solution is better because:

• Minimum interface exposure(non autotvm devs don't need to be aware of the change)
• Random initialization is performed on the device, the current impl actually will results in random number generated on the host then transfering to the device.

[junrushao]

I agree with TQ's point. Also, I am a bit concerned if we really need a C API for this.

[FrozenGene]

I will handle it as comments

[tqchen]

@FrozenGene please followup

[FrozenGene]

Thanks for reminding. I want to complete the clflush pr and model based runtime pr, then handle this pr. Could you help to handle model based runtime pr now?

[FrozenGene]

see comments, we don't need to expose the(non_empty) function as a primitive function, given that it is not a numpy function, and is only used in AutoTVM, instead we can achieve the same goal by

x=nd.empty(..)
random_fill=get_packed_func("contrib.random.random_fill") random_fill(x)

Notably, the above solution is better because:

• Minimum interface exposure(non autotvm devs don't need to be aware of the change)
• Random initialization is performed on the device, the current impl actually will results in random number generated on the host then transfering to the device.

@tqchen I restart this work. This way we could make the random initialization on the device (i.e. AllocaWorkSpace use correct device api). But as far as I know we still can not avoid generating random numbers on host if we don't call specific apis like cuRAND (if we are for NV GPU) . That is to say, we still have to generate random numbers and copy to device. However, we should accomplish the goal we could generate random numbers on the remote cpu (if we are in rpc mode) and copy the data from remote cpu (like arm) to remote GPU (like mali) directly, not the path x86 cpu -> arm cpu -> mali gpu.

[merrymercy]

@FrozenGene Please followup. It is okay to do the path CPU@remote_device -> GPU@remote_device for now, as long as there is no RPC communication cost (i.e. no local_device -> remote device)
I remembered that we tried to do this in our internal repo but failed. What's the problem at that time?

[tqchen]

Ping

[FrozenGene]

@FrozenGene Please followup. It is okay to do the path CPU@remote_device -> GPU@remote_device for now, as long as there is no RPC communication cost (i.e. no local_device -> remote device)
I remembered that we tried to do this in our internal repo but failed. What's the problem at that time?

@merrymercy Our current method is we will introduce one dummy cpu context in the remote and pass the data to the remote target (like OpenCL, CUDA). Previous time we want to do is to generate non empty data in the remote target but failed.

@tqchen 's suggestion we could leverage empty interface and fill the data into the allocated tensor to avoid introducing new non_empty api in the C / ndarray interface and generate random data directly in the remote device. Previous comment is to make sure that we maybe have to introduce cpu like our current way.

I will follow up my pr that move our implementation to the contrib/random/random.cc and turn it on always as our auto scheduler has local builder / local runner also rely on it (not just rpc).

[FrozenGene]

@FrozenGene Please followup. It is okay to do the path CPU@remote_device -> GPU@remote_device for now, as long as there is no RPC communication cost (i.e. no local_device -> remote device)
I remembered that we tried to do this in our internal repo but failed. What's the problem at that time?

@merrymercy Our current method is we will introduce one dummy cpu context in the remote and pass the data to the remote target (like OpenCL, CUDA). Previous time we want to do is to generate non empty data in the remote target but failed.

@tqchen 's suggestion we could leverage empty interface and fill the data into the allocated tensor to avoid introducing new non_empty api in the C / ndarray interface and generate random data directly in the remote device. Previous comment is to make sure that we maybe have to introduce cpu like our current way.

I will follow up my pr that move our implementation to the contrib/random/random.cc and turn it on always as our auto scheduler has local builder / local runner also rely on it (not just rpc).

@merrymercy@tqchen I have updated the code and verified it in the remote cpu / remote mali gpu. We could do CPU@remote_device -> GPU@remote_device directly, not CPU@host->CPU@remote_device -> GPU@remote_device.

[jcf94]

Looks good.
So we will hava a later pr to add clflush & random_fill to AutoTVM & Auto_Scheduler?

[FrozenGene]

Looks good.
So we will hava a later pr to add clflush & random_fill to AutoTVM & Auto_Scheduler?

Yes.

[FrozenGene]

@tqchen@merrymercy gental ping. Code has been updated.

[FrozenGene]

@tqchen@merrymercy@comaniac do you have other comments?

[tqchen]

Thanks @FrozenGene@comaniac@merrymercy

[merrymercy]

@FrozenGene Can you send the follow-up PRs to enable this in ansor and autotvm?

[FrozenGene]

Thanks for reminding @merrymercy. My agenda is completely full tomorrow and weekend. I could do this next week.