Annoyance to Annihilation: Assessing Privacy Risk
•
4 gefällt mir•470 views
Presentation by Rowenna Fielding of Protecture to Data Protection World Forum (GDPR Advanced Theatre) 2018-11-21
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (19)
Ähnlich wie Annoyance to Annihilation: Assessing Privacy Risk
Ähnlich wie Annoyance to Annihilation: Assessing Privacy Risk (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Annoyance to Annihilation: Assessing Privacy Risk
- 1. Meeting all your data protection and privacy needs
- 2. Annoyance to Annihilation Assessing data subject privacy risk Rowenna Fielding | Senior Data Protection Lead www.protecture.org.uk
- 3. GDPR 11 December 2017www.protecture.org.uk Recital 75, 76, 84, 85, 90, 94 DPIA (Article 35) Breach notification (Article34) DPO responsibilities (Article 39)
- 4. classic model: risk = impact x likelihood 11 December 2017www.protecture.org.uk challenges: ‘likelihood’ is largely unquantifiable context dependence varied tolerances
- 5. why infosec models don’t work 11 December 2017www.protecture.org.uk all about C/I/A of information/systems scope is restricted to business risk doesn’t factor risk from things going right no room for rights and freedoms?
- 6. so....what else is there? www.protecture.org.uk we need something new “taking into account the nature, scope and context of the processing...” “...risk to the rights and freedoms of the data subject”
- 7. look at the data www.protecture.org.uk (assuming no personal data breach...) could we do bad stuff with this data? could others do bad stuff with this data? just how bad could we/they be? really bad? quite bad? not very bad?
- 8. significance www.protecture.org.ukwww.protecture.org.uk impact ‘meh’ END OF THE WORLD most data subjects
- 9. another look at the data www.protecture.org.uk Transient Static Permanent easy to change requires significant effort to change can’t be changed low-impact medium-impact high-impact passwords, email alias, preferences, payment card info passport/NI number, postal address, banking details employment/ medical history biometrics, DNA
- 10. keep looking.... www.protecture.org.uk Useful Necessary Critical delay, inconvenience, annoyance significantly impaired function can’t function low-impact medium-impact high-impact passwords, email alias, preferences, payment card info postal address, banking details, geolocation, gov’t-issued ID, medical records, criminal records
- 11. reach www.protecture.org.uk 1. Data subject 2. Data Controller 3/ Data Processor(s) 4. Other Data Controllers 6. General public 1 2 3 4 5 6 5. Other Data Processors
- 12. likelihood
- 13. the Privacy Red Team www.protecture.org.uk
- 14. www.protecture.org.uk END OF THE WORLD only a matter of time your cat meh ridiculously far-fetched Assassins Guild
- 15. summary www.protecture.org.uk look at the data ...stickiness, criticality, reach look at the data subjects ...where on the bell curve? spectrum from best to worst case ...things going right *and* wrong
- 16. www.protecture.org.uk high medium low reach stickiness large volumes of data criticality likelihood special category data some potential for bad effects vulnerable data subjects
- 17. a few notes of caution www.protecture.org.uk you can’t predict everything ....but you can prepare for most things change control, governance, process ...without them, you’re lost computers can’t do the thinking ...GIGO still applies
Hinweis der Redaktion
- Hi, I’m Rowenna Fielding and I’m a professional data protection nerd. I work for Protecture, a company which provides advice, support and assistance on data protection to organisations who are looking for practical help. Today I’m going to talk about privacy risk, specifically to the people whose personal data is being processed. I can’t promise any magic bullets or happy-ever-after solutions, but hopefully I can provide some reasonably simple ways of approaching the question of “what’s the risk?”
- ”....RISK TO RIGHTS AND FREEDOMS....” is a phrase that features quite a lot in the GDPR. since it underpins the intention of data protection law. Have you ever sat down and counted up all of your legal rights? It’s probably impossible to do – you have employment rights, consumer rights, information rights, rights around equality and physical integrity – nearly all based on human rights. You also have certain types of freedoms – of speech, of association, of movement, of religious practice, from torture or arbitrary imprisonment....again, mostly based on human rights. Therefore, impact to rights and freedoms is actually a really big scope in practice!
- When there is a risk to rights and freedoms, data protection law requires you to tread carefully – as you will already know. But what does ‘risk’ mean? In the classic model (albeit very simplified), it means “something might/not happen”. It’s usually used in the context of something bad might happen, or something necessary might not happen. Risk is represented by the combination of impact (what the ‘something’ is) and likelihood (the ‘might’ part) There are lots of models with lots of equations and quantifications of how this applies in everyday life – ask any insurance underwriter! However, when applying this model to privacy risk, ie the potential for impact to the rights and freedoms of the individual data subject, challenges start arising almost straight away. How can you assess impact without knowing the data subjects’ vulnerabilities and sensitivities? It would probably be excessive and unfair to try and find out everyone’s circumstances just so you can assess the individual risks to all of them – and doing so would invoke just as much risk as the processing you’re evaluating – if not more. Impact and likelihood will be change according to the context they are considered in. You can’t just look at the data itself, the way in which it is processed, the reasons why it is processed and the circumstances in which the privacy impact may occur brings in a lot of variables Different people have varying tolerances for individual privacy risk. Dita von Teese doesn’t mind people seeing her without any clothes on, but many of us would probably feel uncomfortable with baring all in public. The Pope is quite happy for people to know he’s Catholic, but in some communities, that could be awkward - or even dangerous for an individual. Some people detest getting marketing emails, others (like my Mum) loves perusing them to find bargains and discounts; and doesn’t mind a bit that her shopping activity is being profiled in pursuit of getting her to buy more stuff (I wouldn’t say my Dad is quite as sanguine though!) So it’s not going to be as simple as putting a number in each variable and acting on the result.
- Many of the risk assessments I have seen in relation to privacy, are based on traditional information security risk calculations. However, I think this is actually a pretty dangerous approach to take because these models, tools and conclusions don’t fit the privacy environment. When considering impact to data subjects, there is a tendency to think mostly in terms of confidentiality and data breaches. There’s a fairly mature industry that’s been dealing with this sort of thing for a while so it’s no surprise that the infosec mindset has dominated. But to focus solely on what happens when things go wrong, and only in terms of unauthorised disclosure or inadvertent exposure; misses the point by a long way. Why? Well, for a start, information security risk is about the confidentiality, integrity and availability of information (which by extension, usually means the C/I/A of systems which give access to and manipulate information as well). While data protection does have a security principle, it’s only 1 of a total of 7, which means that if your risk assessment is constrained by being designed for an infosec scope only, you’re going to miss identifying a lot of risk Information security risk is usually concerned with the risk to the organisation. Thus, if a risk is realised but causes no problem for the organisation then it is a low risk. However, since privacy risk looks at the impact to the individual data subjects, there is a completely different focus – a high impact to the individual which is never traceable back to the culprit organisation is still a high risk, in privacy terms. In infosec risk, if everything is working as intended, then there is an assumption of low risk – assuming that ‘as intended’ includes secure design, configuration and maintenance. However, with privacy risk, everything can be working as intended, and there still be an impact to the data subject. For example; When you browse for products to buy online, you will likely see higher prices quoted if you’re using a Macbook than if you’re on a Windows PC or Android table. The device you are using can be identified, and since Apple gear is so expensive, the assumption is that people who can afford Macs can afford to pay a bit extra for everything else. This is working as intended – but do you think it is fair? A study has shown recently that early-stage Parkinson’s Disease can be identified by analysis of web users’ mouse movements. While this may be very helpful for medical professionals, the information is in the possession of the advertising and data broker industry who have different professional priorities. The risks here are huge, even though the processing is working as designed. Social media platforms allow micro-targeting of advertising content based on demographic categories. These have been used recently to exclude particular groups – people of Afro-Caribbean ethnicity, people who are in lower income brackets or who live in undesirable postcode areas – from seeing particular content since there are higher gains to be made by showing the content only to selected groups. When it’s a pair of shoes; that’s not such a problem – but the recent cases are about advertisements for housing, financial assistance and employment opportunities. Again, this is the processing working as intended but resulting in a huge impact on rights and freedoms for the people who don’t fit into the ‘most profitable’ demographic categories, especially since they would never know that they have been excluded from these opportunities. Information security risk models don’t even begin to touch rights and freedoms. That’s why they are unsuitable as a model for assessing privacy risk.
- ‘Nature, scope, context of processing’ comes up in GDPR comes up a lot. We don't yet have a coherent model for understanding impacts in the short term, let alone any data on the long term because data is infinitely reproducible and once it's out it's out The inability to draw a causal link between one set of processing activities among billions and an impact that is not seen until much later, makes it very difficult to state with any certainty whether an impact will affect any particular individual, or how. When considering impact to data subjects, there is a tendency to think mostly in terms of confidentiality and data breaches. There’s a fairly mature industry that’s been dealing with this sort of thing for a while so it’s no surprise that the infosec mindset has dominated. But to focus solely on what happens when things go wrong, and only in terms of unauthorised disclosure or inadvertent exposure; misses the point by a long way. I’ve been thinking about ways to measure potential data subject impact, and from what I can see, there are a lot of variables involved, such that there’s unlikely to ever be ‘one metric to rule them all’. However, I have a few suggestions which I’m going to share with you.
- The data itself is significant in determining risk – although not definitive. However, it’s the thing that you’re likely to know most about when you embark on an assessment of privacy risk, so that’s where we’ll start. Some data is inherently risky for rights and freedoms, even in the most well-intentioned of hands and without any awkward security incidents. This is the basic premise of the special categories of personal data, however they are not the only high-risk datasets that exist. Volume is risky – even if each individual piece of data is innocuous – or even not itself, personal data – the total aggregate could be a very different thing entirely. For example; Uber IT workers figuring out who’s having extra-marital affairs from aggregate travel histories, dates, times, accounts, locations of users. Bear in mind that ‘bad’ isn’t necessarily about malicious intent – unintended consequences are a huge factor in privacy risk and being bad accidentally needs to be factored in as well. The generation and use of behavioural profiles or algorithmic analysis is high-impact, already, in some countries, these profiles are used for social and political control. In this country, uses have emerged in the areas of law enforcement, education and finance, with unwarranted detriment to many. No attackers, hackers or supervillains involved. For example, the use of software algorithms in criminal sentencing has been shown to result in disproportionately harsher penalties for the same offences being given to non-Caucasians, because the training data which is fed into the software learning model has not been representative of all communities. What might be an indicator of potential recidivism in one person may be totally innocuous in another because data collected for one purpose has been used again in another without consideration of the differences between contexts. So here are the first set of questions to ask when you’re looking at the dataset to be processed. if the answers to the first and second questions are ‘yes’, then there is a risk to the rights and freedoms of the data subject. The answers to the third question will give you a spectrum of impacts and who they are likely to be significant for .
- Consider a normal population distribution. By the way, I am no mathematician, and have no population-level statistics on how people’s rights and freedoms are in fact impacted by the processing of their personal data, but because I had to start somewhere, I began from the premise that by and large, there will be a small number of people at each extreme and most of the rest in the middle. If the data or context of processing is particularly sensitive, then it’s entirely possible that the markers for where the impact lands will appear much more towards the right-hand side. If the data or context is trivial, then the opposite applies. Of course, you can do your own research on what your particular data subjects’ level of vulnerability to risks to their rights and freedoms may be, and shift the markers according to your findings. Ideally, you want to look at the modal average data subject as the ‘best case’ scenario and the most vulnerable data subjects as the ‘worst case’. (That’s ‘vulnerable to privacy impact’, by the way - not necessarily the definition of vulnerable from safeguarding or social care)
- Another factor for potential impact to rights and freedoms in the data itself is how “sticky” the data is. The harder it is to change the data, the more likely it will be that there is a risk to the data subject from processing it, and the more care and attention will need to be taken to minimise this data, ensure it is accurate and up to date, not stored for longer than necessary, and of course, kept appropriately secure. In the event of unfair or unlawful processing, data that is more ‘sticky’ will generally have a higher impact. This is also true in the case of personal data breaches. You can change a password or set up a new email alias with ease. Moving house is rather more difficult but not impossible. You cannot change your DNA, your medical history, or your blood type; so these should always be considered high-risk data sets regardless of how securely they are kept or how trivial the purpose they are processed for.
- Still looking at the data itself... In addition to ‘stickiness’, there is also criticality. The more the data is relied on to produce reliable, consistent effects, the greater impact there would be from it not being accurate, up to date or available. Those of you who have run into problems trying to use the Gov.Verify ID-checking system will recognise the problems that can be caused when critical data is inaccurate or unavailable. Or for example; a mis-spelled name on a plane ticket will cost you a chunk of cash to get rectified but may cause you to be denied boarding if you don’t. How many of you share a name with someone who has been convicted of a crime? If a prospective employer searched your name on the Internet, would a case of mistaken identity result in you missing out on your dream job? Remember, we’re not just talking about what happens if the personal data is unlawfully exposed or disclosed, but what the impact from authorised, everyday processing might be.
- I want to avoid the word ‘confidentiality’ – data may not be confidential, either inherently or in contract but can still present a risk to the data subject’s rights and freedoms. For example, your opinion of your mother-in-law probably isn’t protected by any legal obligation of confidentiality but if it got around, would you be in trouble? What I want to talk about is what I call ‘reach’ – ie, how far the data-ripples are going to travel I think it’s reasonable to assume that the further the data will be travelling, the greater the impact and likelihood to the data subject’s rights and freedoms that may occur – either a a result of the intended processing or of something going Horribly Wrong. Once the data is easily and freely available to the general public, all bets are off – there is an inherent risk which cannot be mitigated after that point. People have stalkers, abusive exes, may live among a community hostile to their religion or sexuality, may come to the attention of an Internet troll army. People acting as private members of the public are not bound by data protection law – therefore processing which extends the reach of the data to being ‘out in public’ should be considered potentially high-risk.
- Finally, let’s look at the likelihood of an impact to the data subject’s rights and freedoms. In my view, trying to calculate likelihood of privacy impact beyond ‘possible/likely/certain’ is too big and intricate a job for anyone but a hard-core data science/privacy researcher to do, because there will always be a huge range of people, circumstances and But remember – we’re not talking about only ‘if something goes wrong’ – your processing may have an impact on data subjects’ rights and freedoms even if there are never any unauthorised data uses or breaches. For example, an algorithm that looks at staff absences and raises a flag for performance management when absences go above a certain threshold would certainly have an impact on the rights and freedoms the employees – not necessarily an unwarranted impact, but potentially so if the system were not configured with sensible exception handling. An employee with a chronic health condition which is known and adjusted for by management should not be subjected to performance management as a result of automated decision-making which is based on the assumption that everyone is usually in good health. In a classic information security model, there would be no risk here, as the algorithm is working as expected and there is no breach of data – however, in data protection terms, there is definitely a risk to the individual’s rights and freedoms – the right to be treated with dignity and equality, employment rights and the right to challenge automated decision-making. The likelihood of some people being adversely and unfairly impacted is higher than zero but the number of people affected will depend on the degree of inclusiveness in the workplace already. So, you need to consider likelihood twice – where does the needle sit assuming everything goes according to plan? Then, where does the needle sit if something were to go wrong? I’m a pessimist by nature, so I’d always be inclined to consider the worst-case scenario at either step. You’re only talking about risk here, not certainty – risks can be mitigated and managed. Whether the mitigation or management approach is proportionate to the risk is a discussion to be had later on in the process. There’s also a question about risk tolerance and ethics to be had – if a small number of people could be devastated but the majority would experience no detriment from your processing, is your organisation OK with the collateral damage? If not, what are you going to do to protect the vulnerable minority? This is where it’s useful to copy an idea from the marketing industry; the use of ‘look-alike profiles’. Without using any single individual’s data as a template, look at the people who are the subjects of the dataset you’re processing. Are there likely to be particular shared factors of vulnerability which you can use as a baseline to estimate impact and likelihood? For example, processing a set of health and social care records will be more likely to have an impact on rights and freedoms than a list of shoe shop customers. Similarly, the processing of email addresses for patients by sexual health clinic will be more likely to invoke a risk to rights and freedoms than the details of who has given a donation to an animal welfare charity.
- If all of this sounds overwhelming for a lone DPO, then fear not because it doesn’t have to be that way. In fact, the process of assessing privacy impact benefits from as many perspectives and as broad a knowledge base as possible. So why not play Privacy Red-Team Have you ever played Cards Against Humanity? Great, then you're already familiar with the mindset needed for this exercise NB: only do this as a desktop exercise – testing ‘what could possibly go wrong’ scenarios on real people is a Bad Idea! Get a group around a table, including representatives from as many different disciplines and business areas as you can haul in. Include a representative for ‘typical’ data subjects if you can! Think of as many scenarios as possible in which the processing (NB: consider access to the data and outputs from the processing too) could damage, distress, exploit, disadvantage, and otherwise screw with the data subject. Assume everything works as implemented at your end as well as stuff getting screwed up. Don't focus on attack or malicious intent until last; unintended consequences are far more common and usually overlooked. Write your scenarios on post-its (other sticky notelet brands are available) When you’ve got your post-it blizzard of ‘what bad things could happen to people as a result of this processing’, you’re ready to move on to the next stage
- Rate the impact of each scenario from ‘meh’ to ‘END OF THE WORLD’, looking across the spectrum from the ‘typical’ to the most vulnerable data subjects. Bear in mind that the data subject wouldn’t necessarily know that the impact originated with your processing, or notice the impact straight away. Grade your scenarios from ‘ridiculously far-fetched’ to ‘only a matter of time’ How difficult would it be to make the scenario reality? ‘You’d need a team of super-ninjas’ to ‘your cat could cause it by accident’ Putting all that together with your matrix of risks from the data itself, you now have a picture of privacy risk, with some insight into how it could occur. You also have some metrics on impact and likelihood which you can use to inform your Data Protection by Design and by Default thinking. Much more fun than filling out forms!
- So to sum up, here’s the high-level view of how to assess privacy risk.
- And because I’m a very visual person, here’s an example of how you could take the output from this session and represent this in an infographic style – much easier to engage with than a 21-page DPIA! As you can see, the priority areas can be quickly identified with this overall picture of the risk profile.
- Some caveats about this approach It’s not based on data science, statistics or anything other than experience in carrying out and supporting others to conduct privacy risk assessments. It doesn’t purport to give you 100% bulletproof guarantees that all risks will be managed within tolerances, because that’s simply unrealistic. However, designing for the majority of risk gives you the majority of protection. Most data protection problems in organisations are not caused by, or fixable by data protection – they arise from deficiencies in organisational governance, change control, culture and process. Unless these are robust, effective and fit-for-purpose, then your risk management tactics will likely fail, despite your best work to anticipate and mitigate potential problems. No software, blinkenlights box, neural nets, machine learning or even the biggest supercomputers in the world can replace the Mark 1 human brain (right now). While there are useful tools to add to your toolkit which can help you to organise, administer and monitor privacy risk; the thinking that must go into assessment of that risk simply cannot be automated with any success. Computers are rubbish at context, and the output of process is heavily dependent on consideration of contexts, cultures, attitudes and human nature. If you treat the privacy risk assessment (or data protection impact assessment, or whatever you want to call it) as a tickbox exercise, a hurdle to jump over then run away from or a challenge to your butt-covering fiction-writing skills then the output you get from it will not be useful. Garbage in means garbage out!
- You can manage risk, but not totally prevent it. However, as long as you can show that you have thought seriously about privacy risk to data subjects, that you’ve identified where and how this might happen, and you have evidence that you’ve taken steps to mitigate this where it is unfair or unwarranted, and explain it where it is neither; then you’re unlikely to experience any nasty surprises.
- Thanks for watching/listening/reading! You can find out more about Protecture using the details on this slide.