When I implemented stacked LSTMs, there was a need to maintain layer-wise parameter matrices in one Parameter, such as
- Wx[0]: Parameter Wx for layer 1,
- Wx[1]: Parameter Wx for layer 2, ... .
Currently there are no good ways to do so,
then my current implementation defines large concatenated matrices and extracts corresponding parts by F.slice(); this is not efficient, hard to read, and suffers from a constraint of the parameter sizes (the size of rows or columns has to be the same).
- Wx = F.concat([ Wx[0], Wx[1], ... , Wx[n-1] ], 1)
- Wx[i] = F.slice(Wx, 1, d*i, (d+1)*i)
If we can maintain such kind of Parameter lists or sets, implementation of stacked models becomes much easier.
When I implemented stacked LSTMs, there was a need to maintain layer-wise parameter matrices in one Parameter, such as
Currently there are no good ways to do so,
then my current implementation defines large concatenated matrices and extracts corresponding parts by F.slice(); this is not efficient, hard to read, and suffers from a constraint of the parameter sizes (the size of rows or columns has to be the same).
If we can maintain such kind of Parameter lists or sets, implementation of stacked models becomes much easier.