Once upon a time, there was a young technical writer who dwelled in an elegant cubicle with high, noise-baffling walls. He was handsome and clever, beloved by engineers and product managers throughout the Valley, and could write like the wind itself.

One dark winter’s evening just before close of business, an ancient crone came tottering over to his desk. In her arms she clutched a tattered collection of printouts of the DR DOS 3.31 manual. “Please, young sir,” the crone asked, “could you trouble yourself to help me edit this poor old woman’s manuscript?”

The young technical writer laughed and refused. “I am far too busy to help the likes of you,” he said, by which he meant too busy reading Reddit. “Begone!”

Once more the crone asked the young man for help with her edits, taking care to warn him that things were not always what they seemed on the surface… and once more the young man refused to help her.

BAMF! With a flash of light and an acrid whiff of toner, the crone’s disguise fell away, revealing her true form: the Goddess of Technical Writing herself.

The young man fell onto his knees, begging forgiveness and sobbing, but it was too little, too late. The Goddess faded away, but not before spitting out her most venomous curse: “May all your software manuals forever be written in TWiki!”

---

So if you’re an unlucky person, you might find yourself stuck with a pile of technical documentation in TWiki or some other baroque “enterprise” wiki. If so, here’s a hacky recipe for getting yourself unstuck.

1. View a rendered wiki page and select the content div out of the HTML, saving each page as an HTML snippet. In TWiki or Foswiki, the content div typically has an ID of patternMainContents. For a small number of pages, you can use your browser’s element inspector to help you copy and paste the content. For a large number of pages, you can try automating this process with curl and some custom script that strips off everything that isn’t the content div.

2. Run all resulting HTML snippets through /usr/bin/tidy.

3. Download and install pandoc.

4. Run all tidied HTML snippets through pandoc, converting them to Markdown, RST, or the output format of your choice.

There’s a good chance the conversion won’t go all the way, particularly if your documentation used any TWiki plugins that rely on JavaScript (like the “Twisty” plugin). Look for sections that didn’t convert properly and fix them up by hand.

The main takeaway is that you don’t want to mess with parsing the horrible native wiki format. Let the wiki do its thing and just get the resulting HTML — that’s something you can work with.

Like everyone else, I make plenty of huge embarrassing typos. Over the years, I’ve tried to discover patterns in how and when I make basic errors.

Perhaps the most common scenario is when I start out with a paragraph that’s correct, and then decide to wordsmith a sentence or clause within that paragraph. For me at least, this is a huge DANGER ZONE moment — there’s an excellent chance that fixing that little passage will actually spawn more errors.

Just to pick a random example, I might start with a paragraph like,

Perhaps one of the most common scenarios is when I start out with a paragraph that’s correct, and then decide to wordsmith a sentence or clause within that paragraph. For me at least, this is a huge DANGER ZONE moment — there’s an excellent chance that fixing that little passage will actually spawn more errors.

Then I decide that the lead sentence would be stronger if I said, “the most common” instead of the more hedging, “one of the most common.” So I change the paragraph:

Perhaps the most common scenarios is when I start out with a paragraph that’s correct, and then decide …

Oops! I forgot to change “scenarios” back to singular. There’s something about going back and focusing on a tiny section of text that makes me forget about overall coherence at the sentence and paragraph.

If this kind of thing trips you up as well, you can do one of two things.

Option A is to do what many professional writers tell you to do: write your entire rough draft without any editing, then go back and do a big editing sweep. Editing entire passages all at once means you’re less likely to make this kind of coherence error.

Unfortunately, I’ve never written that way. I am a writer who backtracks. Always have been, always will be.

So Option B is to cultivate a sense of hyper-awareness anytime you go back and refactor something you’ve just written. This is the approach I take. If you go back and fix up something you’ve just written, the hairs on the back of your neck should rise.

If it helps, you can think of it this way:

• each sentence in a paragraph is like a function
• all the sentences in a paragraph work together like a collection of functions in a small class or module
• if you tweak a sentence that already works, that’s like changing the arguments or the return value of a function that already works. So… are you sure you didn’t screw up something else that depends on that sentence?

You might be wondering, “Are there test suites for sentences?” In fact, yes, there are! These test suites are called “technical copyeditors,” and they’re very expensive to run (even once).

So in lieu of that, your next best option is to go back and eyeball the rest of the paragraph. At a minimum, carefully read the preceding and following clauses for tense agreement, etc. That should catch most of these “regression” style writing errors, but keep in mind that the best defense is always to get another pair of eyes on your work.

Since most of the documentation you produce is going to be hosted as HTML anyway, you might be wondering: why not just cut out the middleman and write all your documentation in plain HTML? Instead of learning some weird intermediate format that transforms to HTML, wouldn’t it make sense to handcraft whatever markup, CSS, and JS that you need directly?

Actually, since most documentation tools and formats range from mediocre to awful, plain HTML isn’t necessarily a bad choice. I’ve seen some beautiful documentation authored in handcrafted HTML. That said, using a more specialized format will make things easier on you. Here’s why.

First, typing out HTML open and close tags is annoying and breaks your writing flow, even if you have a good text editor or IDE. If you’ve used a lightweight markup language like Markdown or reStructuredText, you know the difference. Creating a new paragraph with newline, newline is more natural than angle-bracket, p, angle-bracket. Creating a bullet list with leading *s is much more natural than typing <ul>s and <li>s.

Second, maybe you do want multiple output formats! Are you sure you don’t want PDF? What about man pages? If you want man pages, you’re going to have to invent some mechanism for transforming your HTML into TROFF (a venal sin), or you’re going to have to write the same material twice (a mortal sin).

Third, and perhaps most important, HTML is deliberately primitive and general-purpose. It lacks the semantics that you want for describing technical documentation. For example, when writing a sophisticated book, you might want things like:

• Real cross-references. You want to create a link that points to a section or an example or a table that automatically updates itself when the thing it references moves, or changes its title.
• Fancy admonitions (warnings, dangers, cautions, notes, tips)
• Fancy titled tables and figures
• Fancy titled code examples
• Automatically generated tables of contents, lists of figures, lists of examples…
• Automatically generated glossaries and indexes. (Okay, who am I kidding? Nobody cares about indexes anymore. Sniff.)
• File inclusions (raw, interpreted, syntax highlighted or not, with line numbers or not, …)
• Replaceable text
• Conditional text (generating different aspects of the book from the same source)

… and so on.

I think my bottom line is, you can get away with writing a small amount of documentation in plain HTML, such as a README or a short install guide. But the larger your book, the harder this gets. The pattern you really want to avoid is:

1. Start authoring a substantial book in HTML.
2. Part way through, discover that you need some of the features on the list above.
3. Start hacking those features in with some kind of special ad-hoc syntax. No worries — it’s not too hard, it’s just one or two “special tags” or “macros”…
4. Eventually end up re-inventing a bad version of reStructuredText or Pandoc-flavored Markdown. Except with way more angle brackets.

Don’t be that guy.

I’m pretty sleep deprived this week thanks to my ADORABLE BABIES WHO I ASSURE YOU CAN DO NO WRONG, and so my wife recommended keeping this week’s Tech Pubs Tuesday short and sweet. “How about something like, ‘Use spellcheck,'” she joked.

Ha! Little did she know that “Use Spellcheck” is actually on the list of possible topics I wrote out ahead of time. The joke is on her!

So. Use spellcheck!

Spellcheck is one of your best friends when you’re writing technical documentation. More accurately, spellcheck is your only friend when writing technical documentation.

Certainly grammar check isn’t a very good friend. Grammar check is like that “friend” who keeps passing around links to Truther theories or inane inspirational messages, and man, you’re totally going to unfriend grammar check when you finally get around to pruning your friends list. But spellcheck! When you need to move furniture across town, spellcheck is right there with her truck. Good old spellcheck.

“But Evan,” I hear you say. “Spellcheck doesn’t have all the technical terminology that I need! ” To which I say, “Right-click, Add to Dictionary.”

“But Evan,” I hear you say. “I use vim to write documentation, not some sissy GUI text editor.” To which I say, “I use vim to write documentation too!” And vim has a spellchecker. In fact, the vim spellchecker is… kind of interestingly nerdy. So start using it.

Seriously! Use spellcheck. It’s like having a test suite for your documentation. Or it’s about as close as you’re going to get to testing your documentation this side of the year 2050.

Thank you for your attention. And God help me if there are any spelling errors in this one.

Even for a professional writer, long editing sessions are a challenge. Was I actually editing those last ten pages, or did I nod off around page three?

To help make sure I’ve been editing aggressively enough, I use a simple checkpoint rule:

After editing a first draft, the text should be 25% shorter.

Cutting 25% probably sounds like a lot. How does this work in practice? Let’s take a look at two short real-world examples.

Example 1: jQuery Document Ready

From the “How jQuery Works” tutorial, “Launching Code on Document Ready“:

The first thing that most Javascript programmers end up doing is adding some code to their program, similar to this:

window.onload = function(){ alert(“welcome”); }

Inside of which is the code that you want to run right when the page is loaded. Problematically, however, the Javascript code isn’t run until all images are finished downloading (this includes banner ads). The reason for using window.onload in the first place is that the HTML ‘document’ isn’t finished loading yet, when you first try to run your code.

Breaking this down step-by-step:

1. “The first thing that most Javascript programmers end up doing is adding some code to their program, similar to this” — Already I’m not a fan of this intro, as it doesn’t explain to the poor newbie jQuery developer why “most” JavaScript programmers do this.

   • “The first thing that” — This is throat-clearing. Instead lead off with, “Most JavaScript programmers…” or more accurately, “Many JavaScript programmers…”
   • “end up doing is” — More throat-clearing. Instead, say what the programmers are doing directly.
   • “adding some code to their program” — Too vague. Replace with something more specific, like “wrap their code in an onload function…”
   • “similar to this:” — Also unnecessary. Launching into an example already implies “similar to this.” If you do want to have a transition to the example, you can always go with the classic, “For example:”

2. “Inside of which is the code that you want to run right when the page is loaded.” — The first paragraph now already says most of this, thanks to the “wrap their code in an onload function.” The slightly new information is the “right when the page is loaded.” Delete this sentence, but remember that last bit…

3. “Problematically, however,” — Replace with, “Unfortunately,”

4. “… the JavaScript code isn’t run until all images are finished downloading (this includes banner ads).” — Minor stylistic changes: “isn’t run” to “doesn’t run” and “(this includes banner ads)” to “, including banner ads.” Parenthetical statements are a bit harder to parse, so I try to use them only when necessary.

5. “The reason for using window.onload in the first place is that the HTML ‘document’ isn’t finished loading yet, when you first try to run your code.” — Finally, here is the reasoning we were looking for back in the first paragraph. Why do “most JavaScript programmers” use onload? Because they are waiting for the browser to finish loading the document. So let’s say that, and while we’re at it, move the concept up to the top of the section. “To ensure that their code runs after the browser finishes loading the document…”

The revised passage is now:

To ensure that their code runs after the browser finishes loading the document, many JavaScript programmers wrap their code in an onload function:

window.onload = function(){ alert(“welcome”); }

Unfortunately, the code doesn’t run until all images are finished downloading, including banner ads.

Example 2: Facebook API

From the Facebook Graph API documentation for Achievement(Instance):

The achievement(Instance) object represents the achievement achieved by a user for a particular app.

You can read more about achievements here.

An app can always access achievement(instance) associated with their app with an app or user access_token associated with their app. To access achievements for a user for other apps they require user_games_activity permission and to access achievements for the user’s friends, the app requires friends_games_activity permission.

Breaking this down step-by-step:

1. “… represents the achievement achieved by a user…” — The obvious weak spot is the “achievement achieved.” Eliminating this repetition and changing the clause to active yields, “represents a user’s achievement for a particular app.”

2. “You can read more about achievements here.” — A “click here” sentence! Excellent, a freebie. Just link “achievement” above and then nuke this sentence entirely.

3. “An app can always access achievement(instance) associated with their app with an app or user access_token associated with their app.” — This sentence is hopelessly mangled. Without talking to an actual Facebook engineer, my best stab at this sentence is:

   • “An app” — Shorten to just, “Apps.” This is just a minor stylistic change.
   • “achievement(instance) associated with their app” — Shorten to, “their own achievements.” Since this whole section is about the achievement(instance) object, it’s unambiguous here to just refer to them as “achievements” most of the time. As a side benefit, it’s not clear what Facebook’s convention is around capitalizing achievement(instance), so using “achievements” sidesteps this style issue.
   • “with an app or user access_token associated with their app.” — Change to, “using an app access_token or user access_token.” The “associated with their app” is implied given the sentence that follows. Also, clarify whether we’re talking about a single type of access token (a user or app access token), or two types of access tokens (a user access token or an app access token). Here, I’m guessing it’s the latter.

4. “To access achievements for a user for other apps they require user_games_activity permission” — Break this off into a new sentence, “Accessing a user’s achievements in other apps requires user_games_activity permission.”

   • “To access” — Change to “Accessing.” Another minor stylistic change. The main reason I like it better is that it makes it a bit easier to get rid of the “they” later on in the sentence.
   • “achievements for a user for other apps” — Eliminate the double “for” and shorten to, a “user’s achievements in other apps.”

5. “and to access achievements for the user’s friends, the app requires friends_games_activity permission.” — Create a parallel sentence to the previous one, “Accessing a user’s friend’s achievements requires friends_games_activity permission.” Saying “user’s friend’s achievements” is a little dicey, but I can live with it.

The revised passage is now:

The achievement(Instance) object represents a user’s achievement for a particular app.

Apps can always access their own achievements using an app access_token or user access_token. Accessing a user’s achievements in other apps requires user_games_activity permission. Accessing a user’s friend’s achievements requires friends_games_activity permission.

Conclusion

$ wc /tmp/*.txt
5 67 475 /tmp/facebook-orig.txt
3 43 354 /tmp/facebook-revised.txt
5 85 522 /tmp/jquery-orig.txt
5 42 301 /tmp/jquery-revised.txt
18 237 1652 total

There are a couple of takeaways here.

The first is that you won’t reach the 25% by tinkering at the margins, fixing little issues around grammar and wordiness. You need to dive in and rework entire sentences, paragraphs, and sections. Don’t worry — in a first draft you will always, always find sections that you can delete or radically shorten. The 25% Rule is there to force you to do the harder, more important work of rethinking the text.

The second is that the overall idea is not, “Whoever reduces the text the most WINS AT TECH WRITING!” The idea is only that if you are editing a first draft aggressively enough, then you should probably see a reduction of around 25% or more.

In other words, the 25% Rule is just a rough self-check for your edit. If you’ve crossed the 25% threshold, you’re probably doing okay. There are no “extra points” for going further.

So I’m going to try a new thing in 2013 — a weekly series of short posts on the craft of software technical writing. Welcome to Tech Pubs Tuesday! Today’s topic is code proximity: what it is and why you want it.

There are lots of paths to broken documentation, but one of the most popular paths is to make sure your documentation source and software source live as far apart from each other as possible. In practice, this could mean:

• keeping your documentation source in a separate repo from your software source
• keeping your documentation in a proprietary binary format that only a few people can actually open and use
• keeping your documentation authors as far as possible from your software authors

By contrast, in a healthy project:

• documentation source sits right alongside your software in a /doc directory, embedded inside your source code as doc comments, or both
• corollary: documentation source should be in an open, plain text format
• documentation builds as part of your normal CI process or build process
• documentation authors work right alongside software authors (or are the same people)

In other words, documentation must be part of your project, just like your tests and build scripts. It needs to be in your face every time you view your repo. The last thing you want is for it to be hidden away, in a place you don’t know about, in a format you don’t understand, controlled by people you don’t really know. Code proximity is your first line of defense against documentation rot.

As you might or might not know, I’ve expressed my frustration with the sad state of JavaScript tutorials in the past. Since the universe has not seen fit to fix this problem on its own, I’m finally at the point where I’m working up an actual TOC.

Attention conservation notice: over 900 words collecting some thoughts about a JavaScript book / tutorial for non-programmers.

• There are a tremendous number of people who know at least some HTML and CSS, but who don’t know any programming languages per se.
• Included in that count are people who know at some HTML and CSS, and a smattering of jQuery. This is not a knock on jQuery — quite the contrary, I actually think it’s very impressive that jQuery enables non-programmers to sprinkle in UI effects, etc.
• Still, those who know a smattering of jQuery would also be well-served if they understood JavaScript-the-language.
• Oddly, although JavaScript seems like it would be a natural next language for people who know HTML and CSS, there isn’t a single JavaScript tutorial out there for non-programmers.
• Or at least, there isn’t one that’s any good. As previously discussed, searching for JavaScript tutorials brings up a wasteland of confusing, outdated, incorrect crap.
• It’s actually worse than the situation for HTML, where if you look hard enough, you can find something possibly worth reading.
• Thus, the actual path people follow is: learn HTML and CSS, learn Ruby, PHP, or Python, and then go back and learn JavaScript.
• For experienced programmers, good JavaScript resources exist, if you know where to find them.
• But a lot of what people throw out as introductory resources (MDN, Eloquent JavaScript, The Good Parts) are really introductory resources for programmers.
• Eloquent JavaScript is the closest thing I know of that kinda sorta works for non-programmers. It’s very well written and covers lots of critical topics. I really, really like it.
• The wrinkle is that it’s still too complicated. Chapters 1 and 2 work for non-programmers, but after that the book quickly veers off into functional programming geekery and other advanced topics. I feel safe recommending Eloquent JavaScript to programmers, and possibly science or engineering majors with little or no programming experience, but I just don’t think it works as an introduction to programming itself.
• Writing a guide that teaches non-programmers how to program is tricky, but not impossible. Some people do it brilliantly.
• What the world probably needs is a Learn JavaScript the Hard Way. But Zed Shaw hasn’t written that book yet, and I’m not sure if he cares to.
• In any case, a hypothetical Learn JavaScript the Hard Way couldn’t be a straight port of Learn Python the Hard Way, for reasons mentioned below…
• Problem 1: Printing. Unlike just about every language you can think of, JavaScript in the browser does not have a dead-simple way to write text to the screen. And without a dead-simple way to write text to the screen, it’s awfully hard to get started.
• There’s alert() — um, no.
• There’s document.write() — also, um, no. Any guide that trains newbies to use alert() or document.write() should be set on fire.
• There’s console.log() — better, but I’m not keen on explaining to people that they have to open some inspector or console to see what’s happening. And people who know HTML and CSS really want to be manipulating the page.
• Or, you could ask people to install Node.js. Node.js is great for backend programmers who want to learn JavaScript — particularly because it decouples JavaScript from the DOM. But it’s nice to not have to ask people to install software to get going. More importantly than that, I think people who know HTML and CSS want to be mucking around with the DOM. That’s the touchpoint between programming and what they already know.
• Eloquent JavaScript cleverly works around this whole “print” mess by, well, providing a damn print function.
• In any case, I think there is a way to bootstrap around this. It means we won’t be starting with “Hello World.” Or even, “HELLOSKI.”
• Problem 2: The most interesting early concept to teach is event listeners. That’s a huge part of why client side JavaScript exists, and it has everything to do with why an HTML/CSS jockey would want to branch out into JavaScript.
• But this is not simple. You need to know what a function is — this is a mind blowing concept for non-programmers. You need to know what a callback is!
• Most JavaScript tutorials teach event listeners using the onclick attribute. Partly this is because most JavaScript tutorials were written in 1997 and are total dogshit. But partly this is because using onclick short-circuits the problem of having to grab the element and then attach the listener: complexity is reduced.
• In a Python or Ruby tutorial, you have the luxury of discussing fundamentals like primitive types, etc. before eventually working their way up to callbacks.
• I don’t think this traditional approach cuts it when it comes to client-side JavaScript. I think the reader wants to learn how to click a button and have some interesting stuff happen, ASAP. Then you can start sneaking in numbers, string manipulation, booleans, loops, and so on.
• I think there is a straight line from zero to event listeners that cuts out most of the fat. But I’ll need to try it on some real readers to see if it works.