[AKN #81] Do You Really Need More Data?
LAUGH: One Day It’s A Joke, The Next It’s Reality, LOVE: Early Episodes Of Joe Rogan Experience
Sup homies?
Just flew back from New Orleans on Sunday and boy are my arms tired!
Great start to the newsletter, right?
Hit ‘em with that dad humor out the gate. Drop that hammer. Set the tone.
New Orleans is a great city and I hit it with a great group of friends.
So much fun hitting up Bourbon Street, Frenchman for jazz, and we even did a Crawfish Boil. Really loved the whole experience.
Like I’m honestly so overwhelmed with a sense of gratitude for my friends and the whole experience.
So much so that this isn’t one of those things I’m about to turn into a joke.
I’m just sincerely stoked about the whole thing.
More stoked than the Slopestyle Olympic athletes hitting sick air off a choice jump, landing a 540 on top of the shred shed, and dropping back in goofy on some fresh pow pow.
Highly recommend New Orleans and my friends.
On to the newsletter!
LIVE: Do You Really Need More Data?
"A constant flow of thoughts expressed by other people can stop and deaden your own thought and your own initiative.”
— Arthur Schopenhauer
As a hardware engineer, I work with a lot of data.
We are ALWAYS looking for data to help us understand a problem, whether it be a test result, measurement, or even a picture of how something looks.
Armed with the weapons of statistics and design of experiments, we march forward to discover the Truth.
We are taught to scoff at the subjective takes of mere mortals.
We want to know what is objectively the best solution to a problem.
Abandon your intuition and lean into solving the problem completely with data…but does that always make sense?
The Problem With Only Using Experimentally Generated Data
A common problem I work with in hardware is “Which design is better Design A or Design B?”
To answer that we come up with an experimental design like “run 10 samples of both designs through Test X and assess the difference in pass/fail results.”
The results come back and we get the following:
No Failures for Design A (0 fails out of 10 tested)
Three Failures for Design B (3 fails out of 10 tested)
So Design A is better, right?
Wrong.
Because Fisher’s exact test says the p-value for your 1-tailed test is 0.105 and your significance level was 0.05 so you cannot reject the null hypothesis that Design A and Design B have equivalent performance.
So your scientific training tells you to test more samples — preferably all under the same conditions to eliminate most variability.
So you grab 25 samples of each and run the test again.
Sucks to suck!
This is the world we live in when we make decisions exclusively using data generated inside of a single experiment.
Now, sometimes this is the correct decision.
As we test more units, we find the difference we originally saw in the designs was a mirage.
But other times, the additional testing is a complete waste of time and we knew it the whole time.
Why? Because context matters.
For instance, what if I told you that Design A is a “Brick Wall”, Design B is a “Sand Wall”, and they are both being put through the “Punch The Wall” test?
Those results suddenly seem way more powerful, right?
Maybe we should take into account our expertise in dealing with sand and brick, right?
Again, wrong.
When you live in a world where conclusions are solely governed by experimentally generated data which you personally controlled, the only way to confirm that a brick wall will not be broken by a person's fist is to conduct the punch test 1,000 times…
2,000 times if you want 99% confidence in your result.
Man’s Search For Certainty
This above example sounds stupid, but it happens all the time in both hardware and in life.
In the Information Age, we are taught that we cannot trust our intuitive mind and we instead need to constantly appeal to the rational.
Generate more data! We need more certainty! I can’t “trust my gut.”
We MUST know for CERTAIN that we are drawing the correct answer.
Think of what could go wrong if I got the answer wrong!
This plays out in many different contexts with questionable return on attention.
Yelp Reviews need to be mined and compared to find the best restaurants.
Amazon Reviews need to be read to understand the absolute best squeegee.
And you need to determine your career by deeply understanding the means, medians, and modes of what your contemporaries are working on.
Statistics should dictate your path forward more than your intuitive feel or deep inclinations towards a subject.
Who cares that this restaurant has “good vibes”, Jan K. said the waitress here was kind of a bitch…
Why on Earth would I purchase the Amazon Basics Squeegee when for $4 dollars more I could get the Minterion Squeegee which Rudy said was the “Cadillac of Squeegees”?
And of course, why would I ever paint when there is so much money to be made in computer science?
In all cases, you must be a good devotee of science and conduct a good experiment to eliminate uncertainty.
So we choose to flood our system with more and more information rather than probe what we think and feel.
This rational, scientific approach proved and disproved our scientific intuition countless times and helped us construct the most accurate model of the universe we could possibly come up with.
This is obviously a huge positive for the human race…
However, completely following the Church of Science at all times is lunacy.
By completely following the rational mind and only making decisions through statistics, we:
Expose ourselves to paralysis by analysis and stay standing still for long periods of time as we continue to run more experiments
Explain away all outliers without probing what makes them unique
Remove all the oddities of who we are
We crowd source our personality until there is nothing unique about us left.
So I would say you should consider trusting your gut more.
Not all the time.
Lord knows we don’t need more hippies following their bliss to Bali to become a Yoga Instructor.
But maybe its OK if you eat a meal solely off of the vibes you feel from the outside of the restaurant or explore new things you are interested in just cause it feels right.
At the end of the day, most of your decisions are reversible anyway.
LAUGH: One Day It’s A Joke, The Next It’s Reality
A lot of comedy doesn’t age well.
Like this example of a joke which when read today is very difficult to understand why it is in the comics section…this simply describes something we see every day.
In particular, I like the one of the assistant explaining he is not at his desk because it is so far removed from funny now that I have no idea what they were going for as a joke.
LOVE: Early Episodes Of Joe Rogan Experience
While Rogan’s been in the news quite a bit recently, I found myself enamored by something most people aren’t watching: early clips of his show.
Watching this clip below is mind boggling when you consider where his show is today.
Like when he talks about how weird it is to have 113 people watching his show, you have to laugh at the fact that his viewership now looks like this:
Just a guy who took a weird hobby seriously for over a decade while no one paid attention to him.
Slowly and carefully honing his craft to become entertaining by interviewing his friends.
I wonder what we are paying zero attention to right now, which will fundamentally alter the cultural fabric in 10 years…
What is something that is laughably small, but will become a juggernaut after a decade of consistent hard work?
Maybe its something you are working on!
CLOSING TIME
You don’t have to go home, but you can subscribe here:
Disclaimer: Opinions expressed are strictly my own. Who else’s would they be?
Mahalo,
K.Rapp