KSOM - Simple, but relatively efficient, pytorch-based self organising map

This is a simple implementation of self-organising map training in python, using pytorch for efficiency. This enables to create, train and apply square maps of potentially high dimensions on CPU or GPU.

Three model classes are provided:

• SOM: standard self-organising map
• WSOM: weighted SOM with learnable per-feature weights, optimised through a combination of distance-based and L2 sparsity loss
• MCWSOM: multi-channel weighted SOM, creating multiple WSOM channels with a diversity loss that pushes each channel toward different feature weightings

To install, use

pip install ksom

SOM Example

An example is available in tests/test_img.py for a simple use case creating a square color map of an image. Having loaded the data in a tensor x, the code to initialise and train the SOM looks like this:

fromksomimportSOM, cosine_distance, nb_gaussian
...
smodel=SOM(6, 6, 3, # size of the map and dimension of unitssample_init=samples, # initialised with samplesdist=cosine_distance, # using cosine distance for BMUalpha_init=0.01, # learning ratealpha_drate=1e-7, # decay of learning rateneighborhood_fct=nb_gaussian, # neighbourhood functionneighborhood_init=som_size, # initial neighbourhood radiusneighborhood_drate=0.0001) # decay of neighbourhood radiusperm=torch.randperm(x.size(0)) # to shuffle the dataforiinrange(int(x.size()[0]/1000)):
idx=perm[i*1000:(i+1)*1000]
time1=time.time()
dist,count=smodel.add(x[idx]) # feed the SOM a batch of 1000 pixelsprint(f"{(i+1):06d}K - {dist:.4f} - {(time.time()-time1)*1000:05.2f}ms")

The results on the image on the left looks like the map on the right, where each unit is represented by the colour corresponding to its weights.

Another example is included in the tests/test_cheese.py creating a map of cheeses based on various binary attributes. The results is presented below with on the right the map represented by colours for each unit created through PCA with 3 components for the RGB components of the colour. On the left, a frequency map is given that show how many cheese have each unit for BMU (brighter == more cheese) as well as the name of the attribute most different in this unit compared to the average of the whole dataset.

WSOM Example

The WSOM (Weighted SOM) learns per-feature importance weights alongside the map. It requires a PyTorch optimizer. See tests/test_w_cars.py for a full example.

fromksom.ksomimportWSOM, cosine_distance
...
smodel=WSOM(5, 5, dim,
sample_init=samples,
dist=cosine_distance,
alpha_init=1e-2, alpha_drate=5e-8,
sparcity_coeff=1e-2)
optimizer=torch.optim.Adam(smodel.parameters(), lr=1e-2)
smodel.train()
forepochinrange(n_epochs):
dist, count, loss=smodel.add(batch, optimizer)

MCWSOM Example

The MCWSOM (Multi-Channel Weighted SOM) creates multiple WSOM channels, each learning different feature weightings. A diversity loss (cosine similarity between channel weights) encourages complementary views. See tests/test_mcw_cars.py for a full example.

fromksom.ksomimportMCWSOM, cosine_distance
...
smodel=MCWSOM(5, 5, dim,
sample_init=samples,
dist=cosine_distance,
alpha_init=1e-2, alpha_drate=5e-8,
sparcity_coeff=1e-2,
n_channels=4,
channel_sim_loss_coef=0.5)
optimizer=torch.optim.Adam(smodel.parameters(), lr=1e-2)
smodel.train()
forepochinrange(n_epochs):
dist, count, total_loss, diversity_loss=smodel.add(batch, optimizer)