Skip to content

Rust Migration

tenchd edited this page Aug 5, 2021 · 3 revisions

Investigating the potential gains of migrating our codebase to Rust at some point.
Benefits:

  • Rust is the future.
  • It's easier and more straightforward to write optimized code.
  • Many conveniences of modern languages which C++ lacks.

Downsides:

  • We have to migrate from C++ (maybe not so terrible since our codebase isn't huge)
  • May or may not be good distributed frameworks for Rust yet.
  • Many contributors are more used to C++.

As an informal experiment, I implemented the xor hashing data structure in Rust in an afternoon and optimized it slightly with a friend who's familiar with Rust. On my desktop, it's within a factor 3 of the ingestion speeds of our code on bigboy. I imagine that if we optimized it some more it'd be comparable.

Vector size128 bit buckets64 bit bucketsxor hashing .5 bucket factorRust xor hashing .5 reps (david's desktop)
10^3101840.357 updates/s174429.354 updates/s7156454.406 updates/s3989872.490 updates/s
10^455797.655 updates/s96216.757 updates/s6588353.109 updates/s2975531.218 updates/s
10^525971.327 updates/s45342.837 updates/s6015725.105 updates/s2346723.408 updates/s
10^617071.867 updates/s25876.702 updates/s5036844.517 updates/s2121060.818 updates/s
10^712132.933 updates/s18442.902 updates/s4407694.070 updates/s1780341.003 updates/s
10^89490.997 updates/s14383.936 updates/s4096379.619 updates/s1539481.238 updates/s
10^97652.748 updates/s11602.058 updates/s3673283.474 updates/s1445982.459 updates/s
10^101269.098 updates/sN/A3296869.951 updates/s1272333.978 updates/s
10^11912.666 updates/sN/A3146148.013 updates/s1144202.545 updates/s
10^12805.574 updates/sN/A2888870.913 updates/s1082557.372 updates/s

10^13 through 10^18: 995255.484735287 893804.316069317 867584.356333136 811561.536817464 799080.037052734 729061.895535921

Clone this wiki locally