The ethics of telco fraud
Slides & Questions » tinyurl.com/jbsddz3y Find link to my slides and all the links / references I'm mentioning in google doc. Also drop your questions in here!
Lineup
Deconstruct your proxy
Explicit data design
Assume good faith
Failure-first design
Four primary findings I'll walk you through. These apply to any kind of automation or decision system. Not telco or fraud-specific, they apply to you!
Setting the scene
Starting early 2020, I embedded as an EthicsOps consultant in a small fraud detection team in a telco. ~1yr. 1 DS, 1 Ops, 1 PM, 1 Data Engineer, plus. Using a combo of internal tools and 3rd party payment gateway (ML) for fraud detection. Can't share specifics on features or thresholds. Looking primarily for international revenue sharing but also to a lesser extent service overuse and promotions abuse.
EthicsOps - think like DevOps or SecOps - tooling and process to support ethics. Or like SREs - building for resiliancy but with an awareness never 100% up time - it's never 'done' or 100% perfect.
A balancing act
Fraud is an interesting space in which to discuss ethics. Not only are you (the company) capable of inflicting harm on your customers, you are also under attack by a sub-set of customers who mean you harm. You have to balance the real, often urgent concerns of identifying bad behaviour & halting costs while also acknowledging the possibility of false positives, and considering how you might identify and support any innocent parties who accidentally get caught in the cross-fire. It's a war of attrition - like email spam.
Developing a healthy cynicism about data sources
When we start to feel really comfortable is usually when we discover something can be totally different than our expected interpretation.
Data (interpretation) pride comes before a fall
Anecdote #1: The textivist... (NB Marketing use - overseas texting is EXPENSIVE and outside of most terms of use for personal plans).
Finding #1
Deconstruct your proxy
All data is a proxy - it's an approximation of the thing we actually want to measure. (time allowing - Apple CC example).
Like a reverse persona. Working backwards from the data. It helps us be explicit about the logic leaps we are making. Always asking the question how good a proxy is this data for the thing I REALLY want to know? Don't forget we also measure anti-signals.
Finding #2
Explicit data design
Let's make a distinction between exploratory data science, and load bearing outputs.
Data misinterpretation is easy , and likely under pressure
One of the most interesting things about working with fraud is the pressure it puts on data interpretation in periods of attack. Like putting data science hygiene through the wringer - in unexpected bursts.
Design for cognitive load
Simple doesn't imply you're stupid
Anecdote #2 - the batch delay. Fixed delay in timeframe from data ingest > running feature > alert in slack - got us every time! | Wherever possible, add labels. Be explicit. Make it impossible to misinterpret. Just because your team is smart, technical, doesn't mean you want them to have to be brilliant detectives under time and cost pressure.
Pointers for data interpretation
Avoid DSL
Add units
Support percentiles with absolute (total count)
Make the data recency clear
Finding #3
Assume good faith
A common problem whenever you work with bad actors is to become suspicious, and to start seeing malfeasance everywhere. No need to suspect bad intentions when laziness or incompetance will do.
The antidote to suspicion
Setting the intention to be respectful
The best way to avoid falling into this trap is to be explicitly, deliberately kind and respectful in all our communications. Talking to end-users or customers, designing comms, and how we talk about them inside the team.
bUt wHAt iF iT's A BAd AcTor?
So what?
What is the cost of all outcomes? Being respectful to someone who is trying to rip you off vs speaking harshly to a legit customer by accident? Or disconnecting a bad actor (save $) vs a legit or differently abled customer (lose $). Also consider also reputational harms. Focus on your automation goal.
Keep the moralising out of it
Being 'right' isn't helpful. You don't know that person, you don't know their story.
Tip
Describe behaviours , not people
Describe behaviour (service misuse) not people (fraudster). There's literally no value in applying a moralising title to a person. THere's every possibility of harm, on both sides, if you get it wrong.
👉 You can still set boundaries
Don't mistake my meaning for 'you should let people do whatever they want'. Being kind and calm does not imply letting people get away with bad behaviour or failing to set boundaries in your design.
Finding #4
Failure-first design
Starting from the assumption that the system will sometimes fail, and planning for it.
"Assuming perfect knowledge of the objective decouples the machine from the human: what the human does no longer matters, because the machine knows the goal and pursues it."
-Stuart Russell, Human Compatible
No perfect, bug-free software. Add in data, stochastic universe - more uncertainty. Designing objectives or incentives for ML systems - further complexity. So let's start from the pragmatic view: that our system will fail, and we need to find out when they do.
Designing an escape hatch
Like designing an escape hatch, or adding life boats. You don't want to have to use it, but if you need it you sure hope it's there. Anecdote #3 - finding out about a service mistaken disconnected... from the ombudsman.
Feedback loops must be
Intuitive
Contextual
Timely
I'm a fan of micro-feedback, small and contextual questions that are really lightweight and easy to answer.
Integrate into UI
Confirm your classifications
Collaborative security
Calling back to previous section - don't assume fraud if phishing will do
Just ask
Think the interaction through
Plan for customer support
Capture your learnings
Product/model improvements
Prepare for pushback
People don't like it when you make bad assumptions about them!
Making the implicit explict is going to be uncomfortable
This kind of work can throw up lots of challenges - differences in mental models between what people think they're buying and what you think you're selling. We did some experiments sending comms to provide clearer feedback to people whose service came under suspicion, and allow for them to request support if they think there has been a mistake. We got a LOT of pushback. Some very angry customers.
Findings
Deconstruct your proxy
Explicit data design
Assume good faith
Failure-first design
Let's take a moment to remember these findings we talked through.
— Gif by Barbara Pozzi
A metaphor: sailing in a big cruise liner. When you're below decks, it's quiet, safe, boring even. No sense of motion. When you go out on deck you see the speed, the mass of water displaced, etc... Sometimes we *want* to feel the wind rushing past. I think insulating ourselves from that sensation is unhelpful. We want to feel the wind in our hair, to have a little frisson of fear, not to be too comfortable.
The real game
Trying to catch fraud,
We have more in common with the fraudsters than we might think.
We are driven by the same incentives as the fraudsters - capitalism played out in light and dark markets. Our minds, our assumptions too, are fair game.
Coming soon...
Sweet Summer Child Score
Be a tester? Email laura@debias.ai
Working on a tool which scans for harm to people and communiities by data driven systems, it will be FOSS. I'll be looking for testers to help me kick the tires soon, please reach out if you could help!
