Skip to content

Bug: Spearman correlation incorrect for tied values, ZeroDivisionError on n=1 #14887

Description

@devladpopov

Description

maths/spearman_rank_correlation_coefficient.py has two bugs:

1. Tied values handled incorrectly

assign_ranks() gives tied values sequential ranks instead of averaged ranks. The Spearman formula requires averaged ranks when ties are present.

InputImplementationCorrect (scipy)Error
x=[1,2,2,4] y=[1,2,3,4]1.0000.9505%
x=[1,1,1,1] y=[1,2,3,4]1.0000.500100%
x=[10,20,20,30,30,30] y=[1..6]1.0000.9298%

The worst case reports perfect correlation (1.0) when the true value is 0.5.

2. ZeroDivisionError on n=1

rho=1- (6*d_squared) / (n* (n**2-1)) # n=1: division by 0

Reproduction

fromspearman_rank_correlation_coefficientimportcalculate_spearman_rank_correlation# Bug 1: tiesprint(calculate_spearman_rank_correlation([1,1,1,1], [1,2,3,4]))
# Output: 1.0 (should be ~0.5)# Bug 2: n=1print(calculate_spearman_rank_correlation([1], [1]))
# ZeroDivisionError

Suggested Fix

defassign_ranks(data):
n=len(data)
ranked_data=sorted((value, index) forindex, valueinenumerate(data))
ranks= [0.0] *ni=0whilei<n:
j=iwhilej<n-1andranked_data[j+1][0] ==ranked_data[i][0]:
j+=1avg_rank= (i+j) /2.0+1# averaged rank for tiesforkinrange(i, j+1):
ranks[ranked_data[k][1]] =avg_ranki=j+1returnranks

And add input validation:

ifn<2:
raiseValueError("Need at least 2 data points")

Found during systematic algorithm audit: https://github.com/devladpopov/algorithm-autopsy

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions