Experiment: Quantative evaluation of the unfairness of the user discovery methods for Mastodon

In this book, the quantative evaluation of the fairness of the user discovery methods, also is called unfairness score, is defined as: the geometric mean of the numbers of followers of the users who the user discovery method recommends. In this chapter, the user discovery methods for Mastodon and other decentralized social networks are measured their unfairness score.

In the table below, the user discovery methods and their unfairness scores are shown. The labels A to K correspond with a chapter above in this book.

In this experiment, the user discovery methods were observed at 20:00 to 21:00 on 13th August 2018 in the Tokyo timezone. For the observation of the personalized user discovery methods, the vaginaplant@3.distsn.org [2] account was used. There is an exception that the hakabahitoyo@pawoo.net [3] was used for the observation of the Pawoo's Who to follow panel. For the search engine, the keyword was “ハッピーシュガーライフ” [4] in this experiment. The numbers of followers were observed at 21:00 to 23:00 on the same day.

The unfairness score is the geometric mean of the numbers of followes of the users who the user discovery method recommends. For the user discovery methods which recommend more than 20 users, the first 20 users are computed. For the ones which do not recommend more than 20 users, the whole users are computed. The users who have no follower are treated as to have just 1 follower, because the 0 destructs the geometric mean.

Label

User discovery method

Website

Unfairness score

A

Mastodon Ranking (number of followers)

mastodon.userlocal.jp

1941.5

B

Recommended Followers

followlink.osa-p.net/ recommend.html

1312.7

C

Mastodon Famous Accounts

takulog.info/ mastodon-famous-accounts

1178.3

D

Mastodon Ranking (number of boosts)

mastodon.userlocal.jp/ ?fqdn=boost

1123.7

E

Pawoo

pawoo.net

558.3

F

Home timeline

Embedded

398.3

G

Federate Timeline

Embedded

404.9

H

Local timeline

Embedded

79.8

I

Tootsearch

tootsearch.chotto.moe

134.1

J

User Matching for Fediverse

distsn.org/ user-match.html

39.6

K

Newcomers in Fediverse

distsn.org/ user-new.html

1.8

References

[1] Hakaba Hitoyo, ユーザーディスカバリーメソッドのフェアネスの定量的評価手法の提案, https://gitlab.com/distsn-documents/fairness-quantity

[2] https://3.distsn.org/vaginaplant

[3] https://pawoo.net/@hakabahitoyo

[4] 鍵空とみやき, ハッピーシュガーライフ, https://happysugarlife.tv