What does it means to Optimize, technically speaking?
Unautomatablog Season 1, Episode 1
In 2013, as I was preparing to take a doctoral candidacy exam on the subject of Optimization, I wrote a short document for myself entitled "The Design Optimization Process," that I've decided (finally!) to share in its original form below with all of you, along with the feedback that I received from a well-known subject matter expert in the field...
Wait, wait, wait. Let's time warp to 2015 first through this Mandelbrot Set of fractals.
In the Summer of 2015, I finished my terminal degree at the university, and my focus area was actually something called "Computational Design and Optimization."
I was pretty pumped. Shortly after starting at my new R&D gig working to get government defense contracts, I wrote a proposal; this time I could actually lead the project myself!
NASA basically said "looks great Greg, but where's the Machine Learning?"
I didn't know what Machine Learning was.
Was I supposed to know what Machine Learning was?
I thought that I had understood Optimization, all of its facets, nuances, nooks, and crannies. I thought that I had defended my understanding against expert professors, and that it was more or less complete.
It was a rude awakening to realize that my expertise in "classical" optimization didn't translate to data-driven tools like machine learning, deep learning, and artificial intelligence, at least not tactically, nor even on the surface.
In fact, I found that my expertise didn't appear to translate to creating business value very much at all, anywhere, unless I was OK with being buried deep inside of a government, academic, or commercial R&D lab.
So, within months of getting a PhD, my skills were obsolete.
That was the hard truth. It was time for me to start growing beyond R&D, and towards commercialization (and ultimately, startups). But that's a story for another day.
In short, even though I now had all the degrees, I didn't know what I needed to know. Thus began my lifelong journey to become (and hopefully to stay) #Unautomatable.
So what was this "Design Optimization Process" that I had learned so that I could pass my grad school exams, anyways?
Anyone with interest and availability can check out the full document here, which includes revision suggestions given to me by my original PhD thesis advisor, Dr. Ramana Grandhi (Fun fact: I was actually fired after 3 years of research and luckily got a new advisor that allowed me to finish my degree, but that's also a story for another day.) In any case, Ramana was and is a well-known expert in his field.
Ok, without further ado, the "Design Optimization Process," or better, the "Meta" Optimization Algorithm for designing any system goes something like this...
Design a system
Define potentially better system design(s) than the original
Get started testing any potential new design
Using data gathered during testing, decide which candidate design to test next
Repeat until an optimal solution is achieved
Did I really need a PhD to tell me this?
Do any of us really need a PhD to be told how to experiment? How to leverage the scientific method? How to use trial and error and the resulting data from experience to put ourselves in formation?
I don't believe so. I think we all innately understand how to create something, how to explore and aim at something better, and how to achieve that something better through learning.
This is the essence of optimization. In engineering, in business, and in life.
This is also exactly what machine learning, deep learning, and AI are doing today. However, these data-driven methods do fundamentally augment the art and science of optimization in myriad ways. So we must be diligent when it comes to the details.
We must not have our heads in the sky so much that we over-intellectualize and lose sight of the ground. Concepts and core principles matter, but so does getting down to brass tax.
A brief aside ... 3D printing has also shattered the old-guard frame when we talk about the art and science of structural and mechanical design in the 21st century. But we'll leave that thread loose to be picked up here in a few weeks.
Alas, there will always be core strategic principles, and there will always be tactical implementation details. It's important to know both to fully leverage the tools that will be at your disposal as you make stuff and create value in your own life and career.
So what does it mean to optimize then, technically speaking?
It means to design systems for your life and work that maximize or minimize specific target outcomes, and to do so using the simplest possible solution techniques.
Sometimes the simplest solution requires massive data, but most of the time it does not. Elegant design is most often achieved through simplicity; through the removal of complexity.
Which is why - to me, at least at this point in my career - the answer to the question of "What does it mean to optimize?" is something like:
Via negativa. Figure out what not to do first, then aim at something.
Idea credit: Nassim Taleb
For more on how I developed this thinking, check out my attempt at trying to present it last year at the Advanced Manufacturing Olympics.
And of course, please feel free to share you thoughts, comments, and questions about optimization below on the blog or on YouTube.
Cheers! Talk soon.
May 1st, 2021