Unintended Consequences

Big data apparently can lead to big trouble.  At least that is the word from the Federal Trade Commission.   The FTC is warning marketers and financial institutions that big data and algorithms used to target advertising to specific demographic groups may result in the unintended consequence of unlawful discrimination.   

The FTC report is here.  And here’s one example of a trap for the unwary.   It would be a clear violation of the Equal Credit Opportunity Act to prohibit single women from applying for a prime credit card.  But what if, as a result of some algorithm, the only ads delivered to single women – even women who would qualify for the prime product – are for subprime cards?  Blind reliance on the data could lead to big trouble.  

So how to avoid the problems?  The report suggests several safeguards, but the most basic may be simply this:  “It may be worthwhile to have human oversight of data and algorithms when big data tools are used to make important decisions, such as those implicating health, credit, and employment.”  Seems pretty sensible.