I Prompt, Therefore I Am: The Role of the Designer in the Age of Generative AI

The first time I remember asking the machines for their opinions and perspectives on a design was in 2010 at Lockheed Martin. Lockheed bought my 3D simulation company in 2007, and I created my own skunkworks called Virtual World Labs. We played with AI, machine learning, virtual reality, and augmented reality. One of our projects was trying to build displays for the F35 helmet. We wanted a high-resolution display with a field of view of 160 degrees. Pilots would be able to see their surroundings through the sensors. The plane would become invisible, like Wonder Woman's, with only the sensor data displaying the world around them.

When we took these design constraints to the human engineers, they all shook their heads vigorously and muttered our favorite word, "impossible," and other words like, "not with the materials we have." But we had something they didn't. Hubris, yes. But also a new button on the calculator. We had a new scientific method.

We had machine learning.

We knew that we could ask the machine learning system to develop every possible hypothesis it could contrive, without constraint. We humans would then tell the machines which theories looked promising to us with our puny human brain perspectives. We would prompt them to go deeper and iterate on those. And thus, we watched the machine design a 160-degree helmet out of materials humans had yet to consider in order to achieve the necessary shape and design.

We watched as the AI designed the shape necessary for each lens to achieve the design objectives and then recommended the materials that humans could use.

Fast forward to December 2022. It was the best of times. It was the worst of times for creative professionals. Generative AI had abruptly shifted the landscape beneath us all. Months later, those who create and engage creators are still struggling with the shift, pivot, and swerve strategies and coping mechanisms.

But there is not just one light at the end of this singularity tunnel; many tunnels are winding their way to the future and to many lights. There is a constellation of them. Which light you should go into depends on the stage of your career, your grit and audacity, and the industry or creative path you have chosen or that has chosen you.

We are in the throes of the third (and final) stage of the Information Age, where we create related information and computational systems beyond our understanding and control.

Elon Musk and others are calling for a halt to all development so they can figure out the business models and we can avoid social and financial disruption. People are worried about losing their jobs to AI. And lawyers are being awakened to sue for copyright and trademark infringement—and just general anger and disbelief that their ancient financial models of trading hours for wads of cash are dying out.

What does this mean for creative and design professionals? Read on to hear the dark implications as well as the lights at the end of the many twisting tunnels to our future.

Generative AI: Where Have We Been, and Where Are We Headed?

I think of the Information Age as a chess game. In chess, you have three stages: the opening game, the middle game, and the end game. In the opening game, there are only a few surprises. The middle game is where the game is decided. The middle game has as many potential games as stars in the galaxy.

We have just completed the first stage of the Information Age. In that stage, we created powerful technologies; then, we painfully adapted to them on their terms so that we could make use of them. We learned to program in arcane languages to communicate with computers. We tethered ourselves to desktops, flickering screens, and keyboards, suffering eye strain and carpal tunnel syndrome. In the first stage of the Information Age, the things we made, made us.

In the first stage, our technology asked too much of us and made us work too hard. In the last two decades, we watched schools, hospitals, architecture offices, and companies, large and small, implement technology and completely change their habits, alter their focus, and re-design living and working spaces to meet technology on its terms. We attended training courses for days and weeks. Humans often forget the real aim of their effort: designing better products and working and living environments, educating children, healing people, manufacturing and moving goods, and delivering services that make people's lives better and more productive.

They would end up in a Faustian bargain serving the technology instead of doing the noble, creative things they initially sought to do.

We are awash in data. Information, news, advertisements, apps, videos, and images—all compete for our attention. It is becoming increasingly difficult to sort, sift, and filter an increasingly crowded and confusing Internetworked data space. What should we pay attention to, and what can we safely ignore?

Machines are great at many things. Humans, for the foreseeable future, are still better at others. Combining these two robust computation systems, man and machine, gets you the best results. It does it with minimal effort required from humans. And now that we can prompt and tease generative AI in the direction of our design dreams, we can create faster and more productively than ever before. Having these robust systems serve us personally has ushered in a new age and is changing our relationship with technology.

It gets tricky when our silicon progeny begins to develop their own goals, perhaps not aligned with ours. That is when things could go off the rails.

It is the worst of times.

The one thread I have not seen pulled on much in the coverage of generative AI systems is that, in the first two stages of the Information Age, we have operated comfortably with the idea that certain areas of human endeavor are safe from being usurped by machines. We have assumed that machines are good at computation, storage, and retrieval, but humans alone can imagine, dream, and create art and music. We now have ample evidence that imagination is no longer purely an organic animal activity. Our silicon progeny can dream, imagine, and even hallucinate. It can write poetry and fiction and compose music and art. Machines are now the creators of entire worlds. The plight of creative professionals appears dire as companies begin to prefer the nearly-free creative cycles from machines over humans. But there is another perspective. Perhaps human creative professionals who become adept at guiding machines with design constraints, direction, and prompts will emerge as the new Hans Moravecian “humans in more potent form.”

It is the best of times.

With these new cyborg powers, these transhuman gifts, designers may soon become godlike in their capacity to design and create. In this new potent form with machines in their thrall, designers can harness the entire perspective and power of everything humans have made and everything that the new alien machine perspective can dream of (or hallucinate) to design and create new solutions.

Gone are the days of staring at a blank canvas (whether two-dimensional or three-dimensional or more) stuck in a creative rut. Now we can prompt our AI muses to inspire or create. We are now conductors blending the brass and woodwinds and rhythm sections of our machine inputs into new symphonies of creative output.

We have become cyborg creators of human and machine composition. A stand-alone human without these skills will be considered unnatural, handicapped.

In the spirit of Fred Brooks’s design bible, The Design of Design, the future role of the designer, the architect, the engineer, and the computer scientist will be to define the problem, describe a utility function or goal for the AI, provide some design constraints, and the AI will begin designing.

Occasionally it may ask for human feedback, clarification, or assistance in a new cybernetic feedback loop. At some point in the not-too-distant future, these neural nets may evolve to understand our needs sufficiently that they set their own goals and constraints and design without our guidance, only reaching back to us occasionally for feedback or opinion. We will then be in the age of humans assisting AIs—the final skirmish in the age-old battle of capital vs labor. We will then have to become comfortable with the idea that machine goals will not always be human goals and the singularity event horizon engulfs us all.

Looking back at it now, I realize that the 2010 Lockheed Martin engineering moment where we first asked the AI to help design was a nodal point harbinger of the future. But I believe it will be a best of times scenario for some period of time for design professionals as AI continues to assist us as we work to achieve our goals.