Experiment: Quantative evaluation of the unfairness of the user discovery methods for Mastodon

In this book, the quantative evaluation of the fairness of the user discovery methods, also is called unfairness score, is defined as: the geometric mean of the numbers of followers of the users who the user discovery method recommends. In this chapter, the user discovery methods for Mastodon and other decentralized social networks are measured their unfairness score.

In the table below, the user discovery methods and their unfairness scores are shown. The labels A to K correspond with a chapter above in this book.

In this experiment, the user discovery methods were observed at 20:00 to 21:00 on 13th August 2018 in the Tokyo timezone. For the observation of the personalized user discovery methods, the [e-mail] [2] account was used. There is an exception that the [e-mail] [3] was used for the observation of the Pawoo's Who to follow panel. For the search engine, the keyword was “ハッピーシュガーライフ” [4] in this experiment. The numbers of followers were observed at 21:00 to 23:00 on the same day.

The unfairness score is the geometric mean of the numbers of followes of the users who the user discovery method recommends. For the user discovery methods which recommend more than 20 users, the first 20 users are computed. For the ones which do not recommend more than 20 users, the whole users are computed. The users who have no follower are treated as to have just 1 follower, because the 0 destructs the geometric mean.

Label	User discovery method	Website	Unfairness score
A	Mastodon Ranking (number of followers)	mastodon.userlocal.jp	1941.5
B	Recommended Followers	followlink.osa-p.net/ recommend.html	1312.7
C	Mastodon Famous Accounts	takulog.info/ mastodon-famous-accounts	1178.3
D	Mastodon Ranking (number of boosts)	mastodon.userlocal.jp/ ?fqdn=boost	1123.7
E	Pawoo	pawoo.net	558.3
F	Home timeline	Embedded	398.3
G	Federate Timeline	Embedded	404.9
H	Local timeline	Embedded	79.8
I	Tootsearch	tootsearch.chotto.moe	134.1
J	User Matching for Fediverse	distsn.org/ user-match.html	39.6
K	Newcomers in Fediverse	distsn.org/ user-new.html	1.8

References

[1] Hakaba Hitoyo, ユーザーディスカバリーメソッドのフェアネスの定量的評価手法の提案, https://gitlab.com/distsn-documents/fairness-quantity

[2] https://3.distsn.org/vaginaplant

[3] https://pawoo.net/@hakabahitoyo

[4] 鍵空とみやき, ハッピーシュガーライフ, https://happysugarlife.tv