By Mike Farahbakhshian
In this month’s article, Mike Farahbakhshian shares his midlife crisis with you, with ruminations on aging, legacy technology, and why “new & improved” doesn’t mean “built to last.” Reading Time: 7 minutes. Suggested drink pairing: Taylor Fladgate 40 year tawny port.
Wear and Tear… mostly Tear; or, Where’s Mike?
Happy November, everyone! I’ve heard a few people asking if I was still writing for FedHealthIT. The answer, much to everyone’s dismay and chagrin, is “yes.”
So, what happened?
On Labor Day weekend I suffered the ultimate “hold my beer” moment. I was trying to show a friend the proper technique for flipping a 650lb tractor tire to avoid a distal biceps tendon tear. It worked for the first 8 reps or so, but then…
We then proceeded to purchase a new house. The combination of new milestones and injury left me dwelling on advancing through life. What I’ve gained in experience I’ve lost in resilience, and from what mentors, friends, and statistics tell me, I’m only halfway done. The home inspection snapped me out of my navel gazing. Several times I heard the inspector mutter, “Wow… they don’t build them like this anymore. This was built to last.” His comments stuck with me.
Between my new home, my injury, and my upcoming birthday, I have a lot to unpack about the process of aging, what it means to suffer “wear and tear” and what “built to last” really means in an era of planned obsolescence. While I’ve been referring to physical things – houses and bodies – there’s a lot of lessons we can take away for Health IT.
We’ve established that I’m not getting any younger, so let’s jump right in:
Legacy Technology: Atavisms of a More Trusting Age
At the core of our legal system there are precedents and principles from older systems: Roman, Greek, Babylonian. Although no one uses old or middle English pronouns like thou or thine, or declensions and conjugations, they exist at the core of our religious and legal language. (WITNESSETH: Any use of “Witnesseth” in a teaming agreement or contract.) It’s safe to say that progress is built upon a bedrock of “old.”
In the case of technology, the strata can hide a cybersecurity nightmare waiting to be unearthed by some rogue Heinrich Schliemann type, or worse, a malicious foreign hacker. For example, every device with broadband radio access (smartphones, tablets, IoT devices) has a built-in operating system that operates the Qualcomm baseband radio. What’s worse, this OS operates without modern encryption and is trivially easy to exploit.
Likewise, a Google engineer discovered a very disturbing fact: every modern Intel processor is running a secret copy of MINIX within the processor core, in the Management Engine (ME). If the name is familiar to you, you probably took some Comp-Sci classes. MINIX is definitely legacy technology. Specifically, it is a stripped-down UNIX-alike from 1987 that is used in universities for teaching operating system design. MINIX was the inspiration for Linus Torvalds to create Linux, and unlike Linux, MINIX allows companies to alter the code and not reveal their changes; companies like Intel, who have made proprietary and exploitable changes to MINIX that aren’t open for peer review. (The creator of MINIX wasn’t too pleased.)
MINIX wasn’t designed with security in mind. It is a teaching tool. Putting MINIX at the core of every operating system is like having your new home’s hot water heater made entirely out of Lego Mindstorms, or building a Turing machine out of K’nex. Sure, you can do it, but it’s probably a bad idea. This has led groups like the EFF (Electronic Frontier Foundation) to lead a charge to disable this proprietary management engine. AMD’s proprietary PSP firmware is no better.
I cannot stress the risk and danger of this MINIX-based engine enough. For processor geeks, it runs in “Ring -3” (that’s negative 3, not the Ring 3 that userland applications run in). The lowest ring an OS kernel can access is Ring 0. No user has access to Ring -3. What is running within this hallowed space? I’m glad you asked. Behind the curtain of the holy of holies, the MINIX-based management engine runs:
- A full file system
- USB and other hardware drivers independent of the OS you installed
- A full networking stack including a functional Web server.
A web server. Has Intel gone stark raving mad? Every modern Intel CPU on Earth has an exploitable Web server, ripe for Russian or Chinese hackers, Anonymous, or an AI gone mad to exploit and turn into a botnet of apocalyptic proportions.
What’s worse, these exploitable “sub-kernels” like UEFI, SMM and the MINIX-based Management Engine can harbor exploits that can persist across reboots (in FLASH) and are even effective when the system is powered off. This brings new meaning to the term “kernel panic.”
The trouble doesn’t stop there. Beyond the CPU, many IoT devices and servers are based on firmware-encoded SSH management of the operating system (the one you installed and thought was running the show). Often times these management engines use SSLv1 or outdated RSA encryption. This encryption is vulnerable to exploits like FREAK or Drown. I’ve had to decommission perfectly good blade servers because the Lights-Out-Management (LOM) can’t be updated. Factor this into medical devices like infusion pumps, imaging machines or iron lungs, and you can see the dire straits medical device cybersecurity is in.
There is no glib answer to these problems. They will only get worse as more consumer devices (refrigerators, cars, self-driving cars, and – God forbid – self-driving refrigerators) join the IoT march of progress and create a cacophony of traffic. The bottom line is: the vast majority of our infrastructure, medical IT or otherwise, is built on older technology from a more innocent time. This atavism, especially combined with state-sponsored hackers or malfunctioning/rogue AI, can be a civilization-destroying event.
This isn’t hyperbole. This isn’t limited to IT, or Health IT. The bottom line is, our civilizational infrastructure is based on older, simpler, exploitable rudiments. As long as there are people or programs who choose to, or can accidentally exploit these, we are sitting on a gargantuan powder keg.
For those of you who work on IoT products, chipsets and (medical) devices, please heed these words: make your management layer open source, encrypted and patchable in case of 0-day exploits. Otherwise, you are complicit in letting the bad guys and/or robots destroy the world.
Legacy Processes: Old Dogs, Older Tricks
I’ve spent quite a bit of time warning about legacy technologies, but let’s not exempt process from my harsh judgement. As Healthcare organizations modernize their technology, they will seek to shoehorn this tech into existing processes.
What many IT leaders fail to grasp is that technology changes the way we think about problems and how we solve them. This is the only apropos use of the phrase “paradigm shift.” While there is no validity to the apocryphal Henry Ford quotation, “If I had asked my customers what they wanted, they’d have said a faster horse,” there’s a lot to chew on.
People are currently using 21st century technology to do 20th century things faster and more efficiently. This system will topple eventually. I’m not using my washing machine as a butter churn because we don’t live in an age where we churn our own butter anymore. (I also don’t pop my popcorn in the dryer, either.)
For example, improved predictive analytics exists and is being used in e-commerce. It gets a little creepy, especially when citizens try to do things like hide their pregnancies from Big Data. Yet most fraud, waste and abuse activities are done post-billing. This allows for a huge loophole, like allowing pop-up clinics, mostly in South Florida, to commit Medicare fraud and run with the money. While this leads to very interesting stings, there’s a lot that predictive analytics can do to stop fraud before the billing occurs, rather than post facto.
However, we can’t just jam technology onto an existing legacy process and expect it to work. We need to conceptualize areas where newer technology (AI, analytics, blockchain, [buzzword of the week]) affect processes.
For example, let’s take a look at Natural Language Processing (NLP). When I hurt my arm, I went to the doctor. I described my symptoms and based on his medical training and various mnemonics he learned, he made a series of attempts to diagnose my ailment.
Since doctors have to deal with insurance companies, I was tested for likelier problems before rarer ones, which is why I got tested for a soft tissue injury and not a genetic connective tissue disorder like EDS. Since medicine is still very much fee-for-service, I had to take multiple trips to get an MRI done: once for my shoulder, once for my elbow, and a separate justification and EOB for each trip.
Instead of depending on a doctor’s memory of mnemonics as a stochastic approach to diagnosis, NLP can be used to compare my description, in real time, against a database with the sum medical knowledge of humanity with normalized taxonomy and statistical analysis that takes into account rarer diseases or genetic disorders. That’s the tech.
What needs to change on the human side is that doctors need to emphasize data science over memorization, to emphasize a statistical approach (like that used by Markov chains and in e-commerce) versus a stochastic approach. In recent years, medicine has eschewed rationalism in favor of empiricism, with the belief that the empirical “trial and error” approach allowed more personalized and individual treatment. However, with the advent of genomics and Big Data, it looks like the rationalist approach – treating people with the same statistical confidence algorithms we would treat components of a supply chain – may actually provide for better treatment of the individual.
Technology has changed how we track inventory or diagnose problems in complex supply chains now that asset tracking, RFID and improved self-status-reporting technologies have minimized human error. Now that precision genomics (our own “RFID tag”) and natural language processing (our own “self-status-reporting”) are increasingly viable, we need to restructure medicine from a dialectic conversation between patient and doctor, to a system analogous to those used in data centers and IoT.
Built to Last
In this article I have scared you with the threat of the inevitable IoT firmware apocalypse and proposed the nihilistic theory that the very intimate and human aspect of medicine be improved by turning it into a soulless Amazon warehouse. Welcome to my midlife crisis! Sharing is caring.
The fact remains: “new” is not a synonym for “better”. Old houses have problems – but the ones with good timbers and a solid foundation will be built to last. Floors may settle and bricks may crack, but with a reasonable amount of maintenance, the house will last a long time.
Technology and processes are the same way: when built to last – when future-proofed by designing the expectation of continuous and radical change – they will serve well. But when not built to last, they will fail, no matter how shiny and new.
And when older technologies and processes fail us – like the withered sinews of a middle-aged man – they can be repaired, but the healing process will take time and care. It is the same with repairing our broken IoT and medical device infrastructure, and our antiquated diagnostic and fraud procedures. Like all older things, what they lose in speed, they gain in experience and lessons learned.
Our medical IT infrastructure and processes are an old house with sagging timbers and cracked bricks. The time is now to begin the healing process. One day we will have something that will be built to last.
