I was in a meeting once where someone pitched a really unrealistic idea. I don’t remember the details exactly. But let’s assume this idea depended on pigs being able to fly.
“But how will the pigs fly?”, we asked.
“Oh, we’ll have an algorithm.”
“OK… But, we don’t understand how the algorithm make the pigs fly?”
“I just said, the algorithm will sort that out.”
“But you haven’t explained how?”
“With the algorithm.”
“Algorithms can’t make pigs fly.”
“Algorithm!”
She spoke to two Apple reps. Both very nice, courteous people representing an utterly broken and reprehensible system. The first person was like “I don’t know why, but I swear we’re not discriminating, IT’S JUST THE ALGORITHM”. I shit you not. “IT’S JUST THE ALGORITHM!”.
— DHH (@dhh) November 8, 2019
Don’t devolve your thinking to an algorithm.
Always seek to understand how your algorithm makes decisions. Failure to do so is how unconscious bias gets baked into systems and, in the near future, is going to result in lawsuits (recent changes in EU law mean you can’t use a computer calculation as an excuse for discriminatory conduct any more, and arguing so will likely just get the algorithm banned on top of whatever consequences would have applied anyway).