论文标题
我的结果现在如何概括？在线隐私和安全调查的外部有效性
How Well Do My Results Generalize Now? The External Validity of Online Privacy and Security Surveys
论文作者
论文摘要
隐私和安全研究人员通常依靠通过在线众包平台（例如亚马逊机械土耳其人（MTURK）和多产的）收集的数据。先前的工作 - 使用在2013年至2017年间在美国收集的数据 - 发现有关安全和隐私的MTURK回应通常代表50岁以下的人或某些大学教育。但是，在过去的五年中，在线众包的景观发生了巨大变化，随着多产量的兴起，机器人的存在越来越多。这项工作试图复制有关在线隐私和安全调查的外部有效性的先前结果。我们对MTURK（n = 800）进行了在线调查，对多产量的性别平衡调查（n = 800），以及对多产的代表性调查（n = 800），并将皮尤研究中心进行的概率调查（n = 4272）进行了比较。我们发现，在过去的五年中，MTURK响应质量降解了，我们的结果并未复制有关MTURK响应的普遍性的较早发现。相比之下，我们发现通过多产的数据通常代表了有关用户看法和经验的问题，但没有关于安全和隐私知识的问题。我们还评估了多产设置，注意检查问题和统计方法对在线调查的外部有效性的影响，并提出了有关进行在线隐私和安全调查的最佳实践的建议。
Privacy and security researchers often rely on data collected through online crowdsourcing platforms such as Amazon Mechanical Turk (MTurk) and Prolific. Prior work -- which used data collected in the United States between 2013 and 2017 -- found that MTurk responses regarding security and privacy were generally representative for people under 50 or with some college education. However, the landscape of online crowdsourcing has changed significantly over the last five years, with the rise of Prolific as a major platform and the increasing presence of bots. This work attempts to replicate the prior results about the external validity of online privacy and security surveys. We conduct an online survey on MTurk (n=800), a gender-balanced survey on Prolific (n=800), and a representative survey on Prolific (n=800) and compare the responses to a probabilistic survey conducted by the Pew Research Center (n=4272). We find that MTurk response quality has degraded over the last five years, and our results do not replicate the earlier finding about the generalizability of MTurk responses. By contrast, we find that data collected through Prolific is generally representative for questions about user perceptions and experiences, but not for questions about security and privacy knowledge. We also evaluate the impact of Prolific settings, attention check questions, and statistical methods on the external validity of online surveys, and we develop recommendations about best practices for conducting online privacy and security surveys.
