Should you fail to pilot your own ship, don’t be surprised at what inappropriate port you find yourself docked.
-- Tom Robbins in Jitterbug Perfume
If you probe the interstices of an industry increasingly dominated by Big Business, you’ll discover a microculture of hackers motivated by the mad bliss of invention, surviving on the sweet contagion of creative energy. Employment bonuses mean nothing here; fancy packaging and market share are viewed with contempt if a product lacks art.
Beauty, now that’s the thing—the beauty of elegant code, of a robust network, of a balanced design that “just works” without duct tape and feature bloat.
It is from this culture that the Internet emerged, as well as the Open Source movement. Less obviously, it’s also a diverse community of home-shop machinists, PIC magicians, guerilla solar experimenters, human-powered vehicle designers, robotics hobbyists, amateur radio satellite builders, and countless other independent developers. If you want to see passionate invention without the sloppy overhead of a big R&D budget or the weird constraints of maximizing shareholder value, go find a hacker… someone who gets a techno-boner from circumventing limitations and knows how to get things done.
This has been my world for 30 years—a world where fun is the bottom line and livings are made on the opportunistic spinoffs of creativity, not selling one’s life for a salary. We subsist in the dark matter between industries, trolling flea markets and dumpsters for Obtainium, mail-ordering goodies, making holy pilgrimages to the surplus Mecca of Silicon Valley, re-purposing the detritus of corporate America to our own obsessive ends. Scattered among us are conjurers, alchemists, wizards, lone-wolf inventors, quirky entrepreneurs, larger-than-life writers, and the origins of more than a few disturbing geek stereotypes.
In this parallel universe, the motivation for creating is highly personal. In industry, you can bet that any massive development effort is associated with a business plan—there’s no room for slack in a bottom-line world, and seldom are things done for
fun. But here, you’ll find entire lifetimes given over to chasing quixotic dreams; you’ll see personal fortunes whittled down to marginal subsistence in the name of invention and reputation. Occasionally there’s an imagined pot o’ gold, to be sure, but most likely it’s just a reassuring fiction to keep the
spousal unit calm in the face of demonic focus, Every Goddamn Night Out There in the Shed. No, our motives are usually as guileless as passion itself: chasing daydreams, building tools, realizing obsessions, shattering limits, publishing, earning grins of appreciation from the cognoscenti and accolades from neophytes.
These are things that touch the soul more than the bank account, and there’s definitely a conceit about it—our sense of security lies more in our toolsets than our 401-Ks. We feel sorry for vested employees with their BMWs and well-appointed houses, even as we decorate our labs with rusted hand-me-down office furniture and pay for system upgrades by mining our hardware boneyards through eBay. But money is not the point. It’s the exhilaration of surfing the knee of the learning curve, the almost erotic bliss of a machine flickering to life—catching the spark and glowing while the rest of the world sleeps.
Of course, getting to that point can involve a ludicrous amount of work.
The Microship
OK, so what is it, exactly, that has induced fellow techies to devote their time and energy to a crazy technomadic quest, attracted sponsors, and gobbled up all of my available resources while my contemporaries have been feathering their nests? This project has been a moving target and an all-consuming obsession at the same time, and one of the biggest challenges has been holding on to a central vision that drives the design... even as it changes, sometimes radically, from year to year.
The machine has to satisfy the urges that spawned all this and be intrinsically sexy, yet address practical issues like serviceability and sufficient adherence to standards to ensure the availability of competent help. The underlying fantasy must be potent enough to withstand dead-ends, evolution of technology, and the cyclic wax and wane of
passion. And it has to be beautiful, a bit weird (but never in a gratuitous sense!), and so profoundly enchanting to geek sensibilities that it takes on a life of its own and infuses the very dreams of the participants with visions of the system in action.
Why should you care about some bozo's boat? Simple: in this sprawling website, we are embarking on an exploration of
gonzo engineering, an almost embarrassingly intimate look at how crazy unbalanced people can take an ambitious dream and pull together the resources to make it come true (and then go out and play). You’ll never get a corporate middle manager to admit it, but such lunacy, driven by emotion and other unquantifiable wild cards of the psyche, lies at the very heart of the design process. You can formalize tools and implement procedures all you like, but you can’t fit passion on a PERT chart; trying to do so will repel the very people you need most.
The first step is one of the most fun: indulging in a fantasy rich enough to trigger secret grins of hard-core technolust. That’s the stuff that makes otherwise sensible engineers willing to devote years, if that’s what it takes, to getting it right.
A Touch of Nomadness
I suppose I should begin with a philosophical perspective. After all, the Microship isn’t just a pedal/solar/sailboat, wireless-linked embedded system, or node in a flotilla of like-minded wanderers; nor is it just a telemetry probe, babe magnet, or sneaky way to get back on the corporate speaking circuit now that
BEHEMOTH is retired alongside other silicon-encrusted marvels in The Computer History Museum.
One of the great secrets I’ve discovered is that even someone with stupendously bad work habits (like me) can get a prodigious amount accomplished by applying one simple and obvious technique: keep moving in the same direction for a long time. Unfortunately, that can lead one down the path of
specialization—an essential part of the great symbiosis between those who dream and those who produce. Specialization along with its concomitant skills is obviously necessary to get real work done, but if you’re not careful it can also become a filter through which you see the world, attenuating everything that is not somehow related to your primary focus. Over time, this can cause severe perceptual distortion from which it can be difficult to recover (especially if said specialty ends up, not necessarily through any fault of your own, becoming an evolutionary dead end in a rapidly changing industry).
That’s an easy platitude for a self-proclaimed generalist to spout, but how do we resolve the problem? How do we hold on to a central design objective for a decade or more without becoming like one of those single-issue political or religious zealots who lose the broader context entirely and descend into extremism? It’s much easier to end up there than you might think, especially when you audaciously choose to chase a personal obsession rather than sell 40-hour weeks while hanging onto the remainder for your own sanity-preserving pursuits.
The trick is at once simple and fiendishly tricky: all it takes is caring so passionately about the project that it fills your daydreams, turns trade journals into treasure hunts, induces you to recruit your friends, inspires doodles, and overlays a sense of purpose onto every foray into the backwaters of the web. This is a lot to ask of a job that’s been dumped on you by management, and one of our central messages here is that if this crazy-talk of passion gets you all fired up and chafing at the bonds of a career that isn’t letting you play enough, then maybe some restructuring is in order. For there is simply no way that crank-turning, even by a well-oiled department full of Really Smart People, is going to give you a sustained rush of intense creative obsession; doing that requires a suite of characteristics that are generally regarded as pathological in a corporate environment:
- Enough chutzpah to believe that you are doing something original and important, but the humility to steal shamelessly from the work of those who have preceded you
- Enough schmoozing ability to induce others to buy in to the dream, but the stubbornness to continue believing in your mad quest when associates have given up on you
- Enough optimistic naiveté to interpret catastrophic failures as steps along a continuous path, but the sensitivity to recognize the real gotchas (like your own change of heart) when they subtly appear
- Enough arrogance to ignore the warnings and skepticism of people with far more experience, but the wisdom to shut up and listen quietly to the advice of practitioners in a completely unrelated field
People who behave this way are often described as having attitude problems, difficulty working well with others, and a tendency to jump around and not finish assignments. These are not the things managers look for in employees.
What I’m trying to tell you here is that if you are one of these troublesome folks, you need to shape your environment to support your passions: nothing is more important than removing the barriers that our culture erects around creative madmen, and few companies are willing to customize a job description to allow your brain to go berserk in its own juices. In severe cases, you might even need to jump ship and accept the insecurities that accompany working alone. (On the other hand, if you are in management and are trying to pull off the impossible, then you need to recognize and encourage the hackers in your midst, giving them the freedom to be profoundly annoying and unpredictable.)
All this is simply a contextual backdrop for the real point here, which is that massively audacious feats of creativity fall out of a way of thinking that is much more a
lifestyle than a
toolset. I find myself smirking at books about management and team-building, when virtually every world-changing cusp in the fabric of technology can be at least partly attributed to the obsessive-compulsive behavior of some intense character who broke the rules, dropped out of school, irritated colleagues, jumped between careers, got in trouble, or, as the schoolbooks used to say about the inventors I tended to identify with, “died alone in poverty, an embittered man.”
It seems we keep returning to this theme: a lifestyle of dedication to a mad dream, with everything else shoved aside as necessary to make room for equipment, learning curves, relationships with gurus and assistants, testing phases, and the endless quest for support. It’s not necessarily profitable, nor is it particularly fun (in the amusing sense), but there is something blissful about having a
raison d’etre, a central passion, an unwavering navigational objective that allows every instant of your life to be tagged unambiguously with Distance To Go, Cross-Track Error, Estimated Time of Arrival, and Speed Over Ground. Such clarity may be illusory, but it beats floundering around every day, changing direction on a whim, and questioning your purpose even while working your butt off and looking forward mostly to evenings, weekends, vacations, and retirement.
It’s also no guarantee of success. But even going spectacularly down the tubes feels kind of noble when it’s part of your life’s enduring quest.
Still, I keep wanting to overlay some kind of formality on this. If the Microship is indeed to be a metaphor for gonzo engineering, as I claim, aren’t there a few rules we can apply that are a bit more
useful than saying “just dream it,” like some incongruously successful relic of the 60s who became a crystal-sucker in the New Age fringes of Silicon Valley before stumbling into a founder’s pool during the can’t-fail dotcom boom? Like, it’s all about the fundamental vibrations of your creative energy, man…
Well, um, yes. But if this level of design is indeed a lifestyle, then the closest we can get to “formal tools” is a body of behaviors, attitudes, and hacks. Let’s put on an engineering hat and attempt to consider the problem in that light.
Formal Tools, Briefly Considered
Sometimes I wish I could claim that Microship development had been a tightly managed progression in which, beginning with a vaporous initial concept, we generated increasingly refined formal specification documents, mapped everything onto a PERT chart to establish dependencies, used that to drive human resources and purchasing departments, then underwent a tightly scheduled fabrication and coding phase focused on milestones and design reviews. That’s how big companies claim to do it… and, hey, we even have some nifty project-management software that knows how to convert TO-DO lists into pretty pictures.
During the
BEHEMOTH era, I spent a very interesting afternoon at Trimble Navigation, makers of the bike’s GPS. These weren’t colorful, user-friendly handhelds wrapped around off-the-shelf chipsets back then; they were extremely complex DSP engines coupled with RF hybrid black magic that pushed just about every envelope in the book. I remember being captivated by a massive floor-to-ceiling PERT chart, spanning an entire hallway, the completed boxes bright yellow, the web of interconnections revealing Deep Understanding of the design process and accurate predictions of every step remaining. “I should do this for the bike,” I mused to my host. “It looks like a great tool.”
“Nah,” he replied. “Project management tools assign resources to tasks. You work alone. Just do
something.”
He was right. Even with first-class volunteers and occasional contract help, Nomadic Research Labs is a tiny operation, a
de facto non-profit, beset by overload and bad work habits, constantly challenged by such fundamental issues as demotivation, distraction, and lack of funds. A PERT chart in this environment would be masturbatory, and would presuppose a stable design.
Engineering in a Nutshell
What actually happened was much more organic, and I’ve noted with amusement that, despite protestations to the contrary among the engineering population, it’s typical of the way things usually work in industry. Here’s how to manage a huge, complex project:
- Accept going in that your first tentative decomposition of the fundamental concept will yield an over-simplified TO-DO list, distorted by misunderstanding of key issues.
- Avoiding all the items labeled TBDWL (To Be Dealt With Later) or ATAMO (And Then A Miracle Occurs), dive headlong into the well-defined parts, finishing some of the electronic design so early in the game that it is guaranteed to be obsolete before the physical substrate is built.
- Blunder ahead on the non-obvious parts, getting pleasantly distracted by learning curves and occasional moments of certainty, only to discover basic flaws in your reasoning.
- Now that you are forced to re-think the initial concept, map it onto newly recognized reality to yield a fresh TO-DO list (with new lab notebooks and computational tools to keep things lively) and another cycle of enthusiastic activity.
- Repeat steps 3-4 countless times at varying levels of abstraction ranging from the entire system down to individual components.
- Meanwhile, since technology evolves with frightening rapidity, acknowledge the fact that any computer-based system is such a moving target that if it’s not completed quickly, it will be irrelevant by the time it ships.
- Respond by simplifying the design, further refining your objectives and abandoning dead-end ideas while doggedly pursuing others that have come to represent too large an economic or emotional investment to allow a graceful retreat.
- Compromise here and there, bang out a few things that weren’t on the list, then add them and cross them off to make yourself feel good.
- Get totally sidetracked a few times, and periodically dive into major development marathons to meet public deadlines like trade shows, pulling all-nighters in PFD mode (Procrastination Followed by Despair).
- Announce new completion dates whenever a previously predicted one has passed, and keep driving your PR engine to maintain interest during a process that is a textbook illustration of Hofstadter’s Law (“Everything takes longer than you expect, even when you take into account Hofstadter’s law.”)
Part of this development heuristic is just sloppy management, but it also reflects the way we think. This is why engineering is, at its heart, an art form (and why the average completion time of a homebuilt boat is 135 years).
Perhaps the most interesting thing about this seemingly ugly process is that it’s iterative and self-correcting. Grandiose or stupid ideas may not be obvious during first-pass blue-sky analysis (when the project is glued together by wishful thinking), but it’s another story entirely when it all has to be converted into Clearly-Defined Tasks (CDTs) and drawings that make sense to machinists. Without some kind of closed-loop intellectual process to fine-tune your thinking, it would be impossible to get to the point where you can start using engineering tools to convert fantasies into contraptions.
Trying to shortcut this by starting on Day One with formal design methodologies can have the catastrophic effect of committing you to an ill-defined goal state, whereupon the end result is shaped more by your toolkit than by the supposed objective. That’s why so many products seem malformed, patched, and otherwise inelegant: management loves formal methods and looks askance upon such frivolous notions as approaching product design as a delicate blend of art and engineering. The exceptions, when they occur, are a joy to use. The rest miss the point, no matter how stylish their exterior or sophisticated their underlying technology.
So it appears that designing a system isn’t nearly as rigid a process as typical engineering textbooks would have you believe. Your component choices affect the shape of the thing you’re building; said shape in turn creates constraints that affect your choice of components. Such psychological race conditions can only be resolved by tweaking the granularity knob while adding inputs to your evolving mental model, until the correct solution congeals in a flash.
It’s easy, and here’s how to do it: Prop your feet up on your desk, relax, and form a fantasy of the desired results. Now turn it slowly in your head while calmly examining it from all sides, allowing input variables to float until an unanticipated combination satisfies your psychic fantasy-comparator and generates a flash of recognition. Since all your noodling is naturally saved in a big circular buffer called short-term memory, let this recognition event pre-trigger a snapshot of the conditions that immediately preceded it (before accumulated pondering-propagation delays introduce conceptual drift). There’s your design specification. Take that and run with it.This is probably not an engineering methodology that makes managers comfortable, though it’s a good summary of life in the trenches. There is a pervasive myth that structured methods and sequential procedures, used in isolation, will get you there… but I’ve never seen it work that way. The tools don’t actually start to become useful until you’re quite thoroughly immersed, and that can take weeks of appearing, to outside observers, as if you are loafing.
A Sense of Urgency
Speaking of time, there’s another big difference between gonzo engineering and life in industry. Schedules and deadlines, the X-axis of project management, are anathema to the independent worker. Don’t tell me that I have until Monday morning at 9:00 to hand you a report on the solar array thermal retrofit; I’m still in the wall-staring phase on that one and expect to be here for
days! I might emerge occasionally to troll the web for prior art to steal, get distracted by other parts of the project, or just say “screw it” and go sailing on a friend’s catamaran in the name of research, but a
deadline? Imposing order on the project would send me on a search for something better suited to my interpretation of the term “work.”
Alas, life isn’t like that in a corporate environment, where people actually pay you to behave. Critical-path management, release dates, pre-production prototypes, purchasing cycles, trade shows… there are countless reasons why the long-suffering denizens of cubicles and labs are not given free rein to go with their instincts. But despite the importance of scheduling in coordinating a complex enterprise, there are huge costs involved: design compromises, sneaky shortcuts, employee burnout, kluged patches, bad assumptions, useless documentation, and incomplete testing, just to name a few. This is analogous to sailing: it is well understood that a sailor with no schedule always has fair winds. The people who find themselves calling MAYDAY in a Force 10 gale are usually those who have decided to push their luck for some time-related reason: they’re in a race, vacation’s almost over, the crew has to reach port in time to use a return ticket, or some arbitrary schedule laid out over charts and cruising guides in a cozy den long ago is now affecting the skipper’s judgment.
Working alone and with volunteers on something that will be done when it’s done (and not before), we have the luxury of ignoring the calendar—although with that comes the dangerous temptation to give in to the dreaded
BEHEMOTH Effect (“Hey, here’s a cool gadget; let’s see how we can integrate it into the system!”) Somewhere in there is the right compromise, but we are going to assume that when you’re building your life around the Ultimate Project, schedules are not a factor.
Convenient, eh?
An Economic Aside
While we’re ignoring things, let’s talk about money. From an engineering perspective, this can be even more annoying than time—there’s nothing like “aggressive cost minimization” to take all the fun out of a design. Fortunately, one of the intrinsic features of passionate dream-chasing is that everything else is secondary, and it’s thus easy to justify spending as much as you have (and then some). Combine this with poverty consciousness, and one can get amazingly creative at scrounging. In addition to all the expensive bits from West Marine and McMaster-Carr, the Microship contains thousands of parts that were donated, bought surplus, extracted from dumpsters, horse-traded, repurposed, cannibalized, or fabricated on the cheap. But one issue that never came up was worrying about manufacturability and component cost. There’s a sort of certainty here that is immensely liberating: “This is the most important thing I can possibly be doing, so it doesn’t really matter what it costs to get the job done—I’ll afford it somehow.”
