The Court Speaks, But Who Listens? Automated Compliance Review of the GDPR

Abstract

With the implementation of the General Data Protection Regulation in 2018, the European Union put itself at the forefront of protecting privacy law world-wide. Under the GDPR, data protection agencies may impose fines up to 4% of a firm’s annual worldwide turnover. The largest fines actually imposed now surpass one billion Euro. Still, anecdotal and empirical evidence suggests that many firms violate the GDPR on a regular basis. This could be because such violations may be difficult to detect, or because it may be unclear whether a particular behavior violates the GDPR. This paper analyzes the impact of a drastic example of GDPR enforcement. In July 2020, the European Court of Justice invalidated the EU-US Privacy Shield with immediate effect (“Schrems II”). As a result, many personal data transfers from the European Union to the United States became illegal overnight. We present a unique dataset allowing us not only to observe what firms say about their behavior in privacy policies, but also how firms actually behave. Using machine-learning tools, we analyze the privacy policies of over 7,500 apps on the Spanish Google Play Store and find limited compliance with the Schrems II decision. We validate the quality of our classifier through manual inspection of privacy policies. Using tools from IT security research, we are able to observe the actual personal data traffic flows leaving apps towards the United States after Schrems II. Combining our observations on privacy policies and data traffic flows, our findings on compliance with Schrems II are sobering. A few weeks after Schrems II was decided, only 23% of the studied apps in our sample seem to comply with the decision while 77% seem to violate the GDPR. Over two years after Schrems II, the rate of compliant apps increases, yet we estimate that roughly 45% of the apps are non-compliant. We examine the implications our findings have for the design and enforcement of the GDPR, and discuss how the combination of an automated analysis of contracts and of actual data traffic flows can improve our understanding of how to regulate the digital economy at large scale.