Experiment: Quantative evaluation of the unfairness of the user discovery methods for Mastodon

In this book, the quantative evaluation of the fairness of the user discovery methods, also is called unfairness score, is defined as: the geometric mean of the numbers of followers of the users who the user discovery method recommends. In this chapter, the user discovery methods for Mastodon and other decentralized social networks are measured their unfairness score.

In the table below, the user discovery methods and their unfairness scores are shown. The labels A to K correspond with a chapter above in this book.

In this experiment, the user discovery methods were observed at 20:00 to 21:00 on 13th August 2018 in the Tokyo timezone. For the observation of the personalized user discovery methods, the vaginaplant@3.distsn.org [2] account was used. There is an exception that the hakabahitoyo@pawoo.net [3] was used for the observation of the Pawoo's Who to follow panel. For the search engine, the keyword was “ハッピーシュガーライフ” [4] in this experiment. The numbers of followers were observed at 21:00 to 23:00 on the same day.

The unfairness score is the geometric mean of the numbers of followes of the users who the user discovery method recommends. For the user discovery methods which recommend more than 20 users, the first 20 users are computed. For the ones which do not recommend more than 20 users, the whole users are computed. The users who have no follower are treated as to have just 1 follower, because the 0 destructs the geometric mean.

References

[1] Hakaba Hitoyo, ユーザーディスカバリーメソッドのフェアネスの定量的評価手法の提案, https://gitlab.com/distsn-documents/fairness-quantity

[2] https://3.distsn.org/vaginaplant

[3] https://pawoo.net/@hakabahitoyo

[4] 鍵空とみやき, ハッピーシュガーライフ, https://happysugarlife.tv

PreviousCase study: Fairness of the user discovery methods for Mastodon

Last updated 5 years ago