Easiest way

Almost a decade ago the Happy Hacking keyboard came out.  This was a brilliant product that I can’t believe hasn’t altogether replaced the PC layout and monster keyboard footprint.  I’ve had a few through the years, and I highly recommend them.  They have been at the $70 mark all along though, so I’ve decided to stop buying them, when commodity keyboards are a dime a dozen, even the small footprinted ones.  I’ve also had the legendary OMNIKEY (and maintain it as a keepsake), which is hands down the best keyboard ever made (clickety-click-clack).  But they’re even pre-PS/2, so probably impossible to get working with USB (let me know if you’ve done it!).  So that’s the first and simplest solution if you’ve got the budget: spend the money to get a great UNIX keyboard.

Harder, cheaper way (still easy enough)

But you probably work at various machines, and everyone has a budget.  And sometimes we have to use that silly laptop keyboard.  So now you’re looking at a keyboard reconfig.  If you just want to make CapsLock go away (not a bad idea) and put Ctrl where it’s supposed to be (left of ‘A‘), you can use GNOME to swap them.  Something like: System -> Preferences -> Keyboard -> Layouts -> Other Options -> Ctrl Key Position -> Make CapsLock an additional Ctrl.  There’s also some swapping options, but overall the options there are pretty rigid and don’t get you all the way there.  So don’t bother with this approach, unless you only want to fix the Ctrl key.

What you really need is xkeycaps.  You can install it in a Debian-based system with apt-get install xkeycaps.  Other distros should also make it easy.  It’s not the most intuitive tool to use if you haven’t played with it before (that’s why I’m writing this), so I’ll walk you through what I did with it.  BTW, it’s an ancient tool, from the legendary hacker Jamie Zawinski.

Some keyboard layouts vary slightly from the 104-key standard, but I believe you can get any cheapo or absurd laptop layout to do the right thing with a little experimentation.  We’re going to be reassigning some keys.  It’s crucial that Esc is in a friendly spot — that probably means just left of your 1 key.  Even on a small keyboard with an Esc just one close row up, it’s still too far; you’ve gotta try putting it in the optimal spot.  But then you’ll be losing `~ (or something)!  What a dilemma!  You can’t code without those.  Some will disagree with me here, but I have found the RightAlt key to be a great, easily right-thumb-accessible, unused key, so that’s what I now use for `~.  RightAlt is a sad loss if you’ve got a crappy mouse (since RightAlt-LeftArrow have been the canonical browser back-button), but that may be the subject of another article.  Just get a 5-button or tilt-wheel mouse; recent Ubuntus do the right thing with the extra buttons, in my experience.  So we’ll proceed here to use RightAlt as the `~ replacement.  And we’ll be throwing away the original inaccessible Esc.  That top Function key row is off limits for touch-typists.

You can scramble a bunch of keys around as your see fit for your MO.  Here’s the process I used (as close as I can recall) for the Ctrl, Esc, `~ reassignments I’ve been describing:

• Fire it up from a terminal: xkeycaps
• Just say ‘ok’ on the first screen which should default to the right ‘PC’ layout.
• Hover over the Esc key and notice the KeySym at the top, indicating you’ve looking at Esc.  If you screw anything up, this is the place to see it.
• Move away and physically press Esc and watch it light up on the screen.  Now you can verify any key’s position, which you want to do for any changing keys.
• Make sure all the keys we care about are mapping to the right location.  I found that RightAlt on my wacky keyboard was actually mapping to my number pad’s Enter.  So figure out what your RightAlt is by pressing it.  That’s what we’ll reassign.
• Make the first reassignment.  Right-click-hold on `~ and select Duplicate key.  Then click on RightAlt (or its equivalent).  Now you’ve got another `~ key! (and you can probably figure out the rest).
• Now we can reassign the original `~ to be an Esc.
• Do the same Duplicate key trick to make the original Esc duplicated over `~.
• Finally, do that process one more time assigning the original Ctrl (crazy bottom-left postion) over the top of CapsLock.  Added bonus: no more CapsLock key (remedy that if you want).
• Write out the new layout with the Write Output button (just the Changed Keys is enough).
• Now you’ve got a new ~/.xmodmap-machinename file.  I believe it’s already been activated at this point, or at least xkeycaps has done some magic.
• You’re done.  You might find that some things are not working at this point.  You can always restart xkeycaps and restore to defaults.  But you’ve likely got things right, and just need to log out of X and log back in.  My arrow keys broke when I did this before restarting X.
• Every time you start up X you’ll need to run xmodmap ~/.xmodmap-machinename.  But Ubuntu (or something) actually automatically detected my new .xmodmap and prompted me to ask if I wanted to use it.  That was friendly!
• Use some white-out or tape to relabel your new keys. (optional)

Whew!  That was quite a few steps.  Fortunately, it’s just a one time setup, and subsequent uses of xmodmap at login are trivial (put it  in your .bash_profile or equivalent if not prompted; use $(uname -n) to make it flexible for a shared profile).  I hope you’ve got it working now.  You can leave a comment if you’re having any trouble with the steps and maybe I (or someone) will be able to figure it out.  Breathe a sigh of relief as you re-train your fingers to stop stretching.

I can no longer type without the hack described here.  You can probably alter it for any machine you’re on and any of your own different preferences.  I’m also considering making Ctrl-J a permanent duplicate for Enter.

I should also add a rant for why this article and tools need to exist: Shame on IBM and Microsoft and all the others involved for being such layout idiots and costing the world so much in ongoing CTS healthcare costs.  Especially when the optimal layout was already well-known and widely used.  Remind you of anything (backslash path separator, another ergonomic impropriety; plus countless others)?  It’s really interesting to just stare down at the PC location of Ctrl and Esc, two essential keys probably both in the top 5. They couldn’t possible have been put in less convenient places. Well, maybe they were just taking the QWERTY model of slowing down typists to the next level.  I’m surprised they didn’t think to move Shift up to F5 and Enter over to ScrollLock.

Be sure to thank Jamie for saving your wrist and adding years of screen time to your life.

* Furthermore, I make the typical assumption you’re using some *NIX variant.  Why else would you be here?  But if you’re using Windows or a Mac there are some other tools floating around that will help you customize your keyboard layout; I’m sure they’ll be different from xkeycaps.  Whether hacker or not, anyone can benefit by making these changes.

Why Learn a New Language?

You may disagree or have your own unrelated reasons, but addressing the why seems a necessary start for such a how post.  Here are the reasons I could think of:

1. You want to impress someone. Your present/future boss? Peers? Just padding your resume? This is not a very good reason. Maybe just stick with perl/C++/Java.
2. Pursue a new domain. E.g., I’m heading into R because it’s touted as the best for visualization and statistics, and even has some epidemiologic facilities I hope to make use of.
3. You want to expand your mind. Given your new learnings, you’ll be able to achieve better/faster results with your present favorite language. E.g., learning FP is sure to help you with many other languages.
4. You’re actually going to need to use the new language for real work. You need to work on some app written in it, or you think it will be the best tool for your next challenge.
5. Get connected to a new community. Maybe those relationships will lead to finding startup founders.
6. Have something new to write about. Not a great reason, but I suppose I’m guilty.
7. Feed the hype machine. Maybe you’ve just heard too much buzz about a language and have to know what everyone’s talking about.  This was why I explored Ruby!

There’s got to be more reasons.  What have been yours?

How Long Should it Take?

Depending on your background and the new language you’re targeting, you should plan to dedicate some significant time to learning it. This is ESR on his first run-in with Python:

Not that it took me very long to learn the feature set. I wrote a working, usable fetchmailconf, with GUI, in six working days, of which perhaps the equivalent of two days were spent learning Python itself.

Or you could side with Peter Norvig and devote 10 years to it:

…it takes about ten years to develop expertise in [a programming language].

Maybe Peter only knows perl*, in which case I’d heartily agree, and so would others. Personally, I think a typical competent programmer can pick up a well-designed language in a couple solid months of hands-on study, though this estimate may be off by a couple years.

How To Actually Learn It

We all have to learn a bunch of languages throughout our hacking careers. How do you go about picking up a new one? Get a book? Start hacking? Stare at some existing code? Those are all useful things to do, but I want to streamline the process and get to the best practices. Here’s the checklist I’ve come up with for effectively and efficiently learning a new programming language:

• Click around on the home site. Discover the documentation pages, FAQs, wiki, community, etc. You’ll be back here a lot.
• See what Wikipedia has to say about it. You should see some interesting history and learn about why it behaves and looks the way it does. Maybe you’ll even find something to add.
• Print out the Tutorial and work through it. Mark it up with colors. Type the snippets into your REPL. Change them.
• Read the FAQ. At least browse it so you know something about the common pitfalls.
  
• Read the style guide. If it doesn’t have one, figure out why.  Hopefully you can find editor settings that will enforce it.
• Get a book. Sometimes it takes a couple. With Perl you’ll need eight tomes, all of which will be witty, yet combinedly inconsistent. With Python you’ll need one. (some will argue which one that is, many are great).
• Avoid certain languages. If you can’t find a book that dedicates fewer than 100 pages to wholly describing the language core, then something is wrong with the language. C++ and perl are violators of this, and prime examples of languages to avoid in all capacities, at all costs. You will waste all your energy on learning and remembering their quirks, instead of focusing on problems to solve. I can’t believe some institutions (my undergrad!) have used these to teach OOP. I would go so far as to call their paradigm QOP, or “quirks-oriented programming”. (WOP — “warts-oriented programming” — isn’t undeservedly harsh, but maybe not PC.). They will damage your thinking process, waste years of your programming career, or destroy it altogether, and you will actually become less competent for having ingrained their quirks into your cognition.*
• Work through the examples in the book. Highlight and scribble all over it. If the book is more than 600 pages, then either the language is crufty or the book is poorly organized, and you might not want either.
• Start collecting miscellaneous pages from around the web and bookmark and tag them appropriately. Look for language luminaries that have created nice bookmark collections.
• Hang out on the IRC channel. That will be irc.freenode.net.  The traffic on some of these is overwhelming, but you can quickly get a feel for what are the tricky corners of the language.
• Find a local user group to meet people and get connected. You can even get a feel for the language by surveying the attendees. Watch for excessive drool! If a SIG doesn’t exist yet, then start one. If no one is interested in joining, move to a bigger town or find a new language, or try to associate with one online.
• Join the users (and announcements) mailing list. You’ll get to see lots of code written by others and see solutions to common problems.
• Watch some videos (Google tech talks) and slideshows.
• Play in the REPL. Get a better REPL if available (ipython). What, no REPL? Beanshell doesn’t count. Bye-bye, Java!
• Try the debugger. See if there are better ones.  Can you dive right into it from crashing code?
• Get the tool suite. Get familiar with the tools and understand why they’re necessary.
• Explore the vaults. Examples are Python’s PyPI (Cheeseshop), R’s CRAN, perl’s CPAN (though CPAN has similar issues to its books). Having extensive add-ons is a good language health indicator. Also note how difficult their packages are to install.
• Write a simple tool. That will season you in making use of important facilities. A good example is grep, which will exercise regexes, getopt (Erlang’s was hard to find), command line, strings, files, etc.
• Blog about your experience. Trying to describe your learnings to others will ensure that you understand what you’re talking about.  Do I?   Learn by teaching can be even more effective than learn by doing.
• Find the right interpreter/compiler. Some languages (Lisp) provide many.
• Install it, and its documentation.  apt-get install ... Not available in apt?  Then it’s too new, or you’re seriously bleeding edge, or it’s not viable.
• Figure out how to use the documentation systems. Manpages, info, –help, REPL-tab-completion, special tool (pydoc, Edoc, perldoc))
• Make sure your editor is equipped. What? it prescribes an IDE for you? Later, Smalltalk (Squeak)! Mess with your editor until it’s working as intended. Syntax highlighting, auto-completion, auto-indent, macros, etc.
• Learn a bit about its history. Why was the language created.  For what niche?  Are its strengths and weaknesses reflected in that history?
• Make sure it’s open. I shouldn”t go into this again, and I realize it’s a loaded statement, but I say avoid anything that doesn’t have an excessively open community and standards body around it.  I’m still teetering on the Flex (Adobe) fence, but anything that Microsoft has had its hands in developing is simply poisonous.
• Compare to a language you know.  What’s it most similar to?
• Take a class. This is often not necessary, but if you’ve got some college nearby they may offer something helpful.
• Get certified. Some languages have (silly) certifications.  The main benefit of them is that there is training material built around them that help you focus on what someone thinks are the important pieces of the language.  Doing this with Java helped my focus quite a bit (though I’ve since left the language behind).
• Watch/shadow a pro. This can be the absolute fastest way to see what to do, or what not to do.  Careful that you choose the right pro.
• Study the idioms and quirks. All languages have some quirks, even Python.  It’s best to know about them early.
• Find and explore a project source base that is known to be well-written. Dive into the code.  Try making some changes.

I’m not suggesting you need to do all these things for every language you explore.  This list could even serve as an evaluation for a new language you’re just considering.

Bonus: Go Deep

If you’d like to move on to the next prescribed level, here are some bonus activities:

• Find a bug in a library, and submit it to the BTS.
• Fix a bug and submit a patch.
• Understand how to mark up your code for document auto-generation.  Write a manpage/vignette.
• Check out the source using the VCS and build the compiler/interpreter yourself. If this is hard, you might consider if your apps to-be-written in this language can ever be deployed.  (Huge language bonus if they’ve got it together enough to be using a DVCS.)
• Try your app on multiple OSs. If you can get it to work on Linux, Mac, and Windows (at least Cygwin), you’re probably safe. Even a web app should have some portability since you never know what future co-developers might think they need to do.
• Start or join a real project.
• Make use of shared memory or message passing or whatever concurrency facilities are provided. Measure the quality of the language by how long it takes to figure out how to do robust parallel programming. Erlang takes a week. If it’s years, like most, you may want to move on, or read the next item.
• Integrate with another language, particularly C.  If this is impossible, it’s probably a showstopper.  Compare to Python and R as benchmark examples.

If you can’t accomplish all these things with the new language you’re considering, just move on to the next. There are plenty to try!

Note that these are all great things to do when getting started with a new language. However, you probably don’t want to be hanging out on IRC or mailing lists forever, or writing excessive posts about your fascinating new learnings, or reading too many books. I’ve fallen into all those traps.  The best advice I can give you (and myself) is to get those hands dirty!

What other things do you do to get comfortable with a new language?

* Notes: [1] Peter is actually pretty bright, and I was joking about perl. I sat on the floor right next to him last year at Startup School, without realizing who he was until he excused himself to give the next talk. [2] Wow, Stevey; that second glass certainly does get the juices flowing!

Here are some great things I’ve discovered about R:

• Functional. Although I’ve dabbled in Erlang, Haskell, and ML, and grown towards using Python in various functional ways (Mertz, AMK), I can’t say I’m fluent in any sanctioned functional language (okay, I just added R to that list, but it should have been there: “R’s functional parts come from Scheme.”)  I want to continue “breaking my brain in useful and creative ways”, and R fits the bill for completing this repertory need (I’ll probably head back to Haskell if/when a strong need for concurrency arises).
• Statistics and data analysis. The Numerati and other sources say statisticians rule the world.  My eyes have been opened and I believe they’re right, so I’m working back my math chops.  It would appear that R’s statistics packages are more expansive than Python’s, and more is built into the language itself.
• Mathematical learning tool. It’s been a decade since I finished that math degree, but now the investment is finally paying off… I, uh, hypothesize.  I’ll blame the hiatus on not having found very interesting work to do until now — another reason to be self-employed, and why I’m loving life this year.  Anyway, I’m already getting re-versed in stats just by starting to use the language.  It’s looking like playing with R is a much better way to learn statistics than muddling through textbooks and paper-based homework.
• Graphical. Don’t know where to start.  There are so many graphical tools for R it is mind-numbing.  I’m starting to look at RGgobi, but there are lots of others to get acquainted with, including what’s built in.  I’ve worked through a couple graphical tutorials, and they seem to just magically pop up amazing graphs without having installed anything.  Try: > demo('graphics') Notice how few lines of code are doing all that. Wow!
• Geospatial. I’ve got a need to be plotting data on maps in a variety of formats.  I’ve found a lot of ways to do this, but R seems to be the lightest, and very capable.
• High-level. Everything is a data structure.  Operations applied to variables are done across the whole set with no loops or treatment of individual items.  The syntax appears to be a bit richer (higher level, more sugar) than found in Python.
• I/O. R seamlessly slurps table-oriented text files for processing.  Output is also automatically formatted in nice text tables.  A mini-book (PDF) describes this and much more.  My next learning task is to start interfacing with PostgreSQL.
• Kind of Python-like. I’ve started outlining another article that I hope to actually write someday called “R For Python Programmers” (since I can’t find such a guide).  Isn’t Python the gold standard to which great books make comparisons these days?  I won’t duplicate that here, but simply say that I can’t believe how comfortable the syntax feels coming from Python nativity.  And R appears to integrate very well with Python (this probably being the more important point).
• Best REPL evar! I start evaluating any language simply by firing up its REPL and comparing its facilities to those of IPython.  R’s REPL is on par with it (readline editing with vim-mode support, tab completion for everything, extensive help), and even has some extra niceties; e.g., function parameter tab-completion.  While getting started, note that the help system uses ? and ?? prefixes instead of IPython’s suffix notation.  IOW, use ?topic instead of topic? I’m getting the impression that a common workflow is spending lots of time in the REPL working with files and graphics.  This is probably the killer feature that enables me to quickly get up to speed.
• Documentation. The pages aren’t pretty, but there is a mass of info on the R site.  And AFAICT R is the foremost language used in recent Statistics textbooks.  There’s also a free 100-page (a good length, compare to Python Tutorial) intro book.  It’s horrible for non-programmers, and less than perfect for non-statisticians, but will get you familiar with some language features.  Appendix A offers a nice REPL walk-thru of language features.  I can’t find any Python books with much treatment of numerics, except for this one which seems to touch on some but shares space with language basics.
• CRAN. Incredibly diverse libraries (list of packages) for statistics, graphics, and even Epidemiology (probably overkill for my present needs).
• Mature and well-designed. R has been growing as the de facto FOSS statistical/graphical language for over a decade (inception in 1993, in the golden age).  It has grown up from the learnings of its ancestor S (shouldn’t R then be T??; I guess similar to C->B->A progression), which came onto the scene circa 1976 (a good year .
• Widely used. A survey of university statistics courses and Public Health curricula show R as a prevalent tool; e.g., UC Berkeley and Iowa State.
• UNIX-friendly. I was humored to see that R is more apt to borrow names from UNIX commands than from other languages.  The commands for managing a namespace are rm and ls — easy for me to remember.   And I’m glad to see it has good roots.
• Trivial Ubuntu installation. Try this: apt-get install r-recommended r-cran-<tab><tab>
• Script-friendly. Rscript enables R to act as a scripting language.
• PDX-visualization group. This month we’re starting up a group to discuss visualization technology/advances, and R will be the primary language under discussion.  My statistician friend, Ed, won’t stop talking about R, and he’s someone I’ve come to listen intently to.

It’s a bit early to be writing an article on a language I haven’t done much with yet.  But I had to explore what’s pushing me towards R.  Looks pretty compelling now, so I’m very close to diving in (actually using it in a project).  I’ve sprinkled a number of resources throughout this post which I hope will be helpful for newcomers to R (including myself).  I should also mention that the best articles I’ve seen on introducing R are by David Mertz (a very capable Pythonista): Statistical Programming with R, Part I and Part II.

I can’t ignore what’s been growing out of  Scipy, especially the maps.  And matplotlib’s gallery is incredible (and I have some of them working).  The above items are all great features of R, but if I can accomplish them nearly as well in Python I probably shouldn’t invest too much into R.  At this point it does appear that R offers some facilities beyond Python.  Have you worked with both Python and R and found compelling reasons to prefer R?  For which types of tasks?

My pain points with Python today are in analyzing query data (ad hoc in loops and heuristics), and creating a bunch of different formats to send to various third-party visualizations.  My analysis is going to need to get more sophisticated, and I’d like to be able to look at data visually quickly with less overhead.

I’m probably overstating the investment required in getting up and running with R; it’s supposed to be easy to pick up.  I’d really like to get to being comfortable with applying non-trivial statistics, which means working through some R books.  I’ve made it this far (and now so have you), so I’m going for it, hoping it will be a pretty quick learn.  Please feel free to share your experiences with learning and using R!

• $ apt-get install graphviz python-pydot python-pygraphviz # Python/Ubuntu friendly
• $ man graphviz (will lead to many other pages)
• O’Reilly tutorial by Michele Simionato (some other tutorials are not great)
• Some flickr example images, some with sample code included
• Simplest first use: $ dot hello.dot -Tpng -o hello.png && firefox hello.png
• Use from Python
• Plug Graphvis into your wiki (MoinMoin)

I (with Tim) am also planning to show some of our progress on visualizations for uGraph, so be sure to show up for the meeting.  And what a fantastic group of folks: many entrepreneurs and data miners.  Should be a great after-party!

After some hunting around I found a help article at Webfaction describing the process of sending mail from a script (with Python, nice! give it a thumbs up; those guys are great).  I could easily enough put some code like that into a function in a utility module to be called from various places, but it’s kind of ugly and repeats some data multiple times and requires a couple more imports.  On a development box I got Django’s handy send_mail function to work, but even with the described smtplib setup work on the webhost, the Django convenience was not going to fly.  But shortly into that Django article you see some setting parameters that are not part of the default django-admin-generated skeleton.  I tried setting these like so:

if platform.uname()[1].lower() == 'myserver.webfaction.com':
    EMAIL_HOST = 'smtp.webfaction.com'
    EMAIL_HOST_USER = 'my_user_name'
    EMAIL_HOST_PASSWORD = get_my_password()

And now I can simply send myself emails by making calls to send_mail, like this:

from django.core.mail import send_mail
...
send_mail("New beta user signed up!",
    "%s\n%s" % (email, text),
    "beta@mycompany.com", [ADMIN])

I’ll assume you’re on a Debian-based/derived system and have access to apt-get.  If not, you can probably install all of these by downloading individually or relying on easy_install/PyPI.

Python

These are the resources to get you familiar with the language, culture, and communities.  I personally like to start with books.  I’ve read many, but thanks to Python’s clean/simple design, only a couple are actually necessary to make you proficient.

Install: sudo apt-get install python python-ipython python-setuptools

• python -c 'import this' (Zen of Python)
• Python glossary
• Python tutorial
• Python Standard Library
• Python wiki
• PyPI
• Book: Python in a Nutshell (best Nutshell book ever written, regexes and libs coverage pretty comprehensive, but hoping for Py3k coverage soon!)
• Alternative free/open book: Dive Into Python
• Book: Python Cookbook (also online ASPN Cookbook)
• PEP 8 (style guide)
• Programming FAQs
• Editor configuration (vim: set ts=4 ai sw=4 et tw=72; research equivalents your favorite editor; emacs good alternative)
• ReStructuredText (ReST)
• Python mailing list
• Python IRC
• Find a Python user group (ours is Portland)
• The here/near future (Python 3)

Tools

If you only get one thing out of this article, let it be this: Use ipython to interactively discover the language and libraries. It’s the most powerful REPL of any language I’ve encountered, and is what really makes Python rise to the top for learnability.

Install: sudo apt-get python-ipython pylint pyflakes pychecker mercurial python-nose pydb python-coverage python-sphinx

• Interactive play: ipython (try: foo.<tab> and foo? and discover everything!)
• PyPI installables: easy_install
• Testing: nose
• Debugging: pydb (extended Python debugger)
• Lint/correctness: pylint, pyflakes, pychecker
• Code coverage: python-coverage

I’ve skipped some really useful tools that go beyond the essentials.  If you’re going to become a Python zealot, you’ll want most of your tools to be written in Python.  These might include Mercurial, Bazaar, Trac, MoinMoin, Sphinx.  We still need investigate the use of Selenium and Twill for web testing.  (We’re now using PostgreSQL for all our database work for several reasons, but one of the most significant is its REPL similar in introspective capability to ipython.)

Django

Our choice of Django was not an easy one, but after some time with Pylons and TurboGears (and even RoR), it was the winner.  It’s also arguably sort of the official Python web framework and does appear to have critical mass (AFAICT).  The Django website lists many of the following resources pretty clearly, but I have found these to be the most useful, and so worth enumerating here.

Install: sudo apt-get python-django python-psycopg2 python-sqlite3

• Django tutorial
• Django cheat sheet
• Book: The Django Book (especially chs 3-5,12)
• Django documentation
• Django Master Class presentation
• Django FAQs
• Django mailing list
• Testing Django page
  
• Long Django testing discussion
• Django Debug Toolbar
• Especially informative blog: B-List
• Most Django blogs

Please include any of your favorite resources for Python/Django newbies in the comments.

UPDATE: I now do almost all my reading on an Android, using Aldiko or Kindle apps. Amazing what two years have done for books! I haven’t printed a book according to this post in a while. I’m finally publishing this post (sans the pictures that would have been helpful) in case it’s still useful to anyone to be able to print sub-120-page e-books on their own printers. And I’m posting the script as a gist in case anyone wants to use it.

So I did my usual routine of over-thinking a problem (held probably only by myself), and decided I should be able to print TBB just like a publisher.  It should have linear flow, be bound in the middle (with a couple staples); basically equivalent to a real book.  Hmm, now how to lay out those pages?  Just a SMOP, right?  Almost.

After messing around with various printer utilities (cups, lp, lpr, lpoptions, foomatic, and more) and being teased that I was close, I ultimately was not able to invoke the printer the way I needed from a script.  But even if I could, it would be a pretty system-specific solution.  Probably better would just be to print out a list of page numbers that could be pasted into any print-wizard’s “page range” prompt.  I don’t need to automate this completely (yet).

If you’ve given any thought to what such a page arrangement would look like you’ll realize it’s slightly complex.  But I got a little script working that does the job well.  I’ll spare the details unless you ask for them.  Download it here.  Basically, the order for an 8-page book is: 8,1,6,3,2,7,4,5.  You can print out your pages in that sequence, cut them vertically in half (hopefully you’ve got a honking-blade paper cutter, but scissors will do), put the pages on top of each other, and squeeze off two staples in the middle for the binding.  Wrapping in 100-lb cover is optional.

There’s one remaining hurdle.  Say you want to print a 7 page book.  Now the order is *blank*,1,6,3,2,7,4,5.  Does your printer allow that?  How about 9 pages?  b,1,b,3,2,b,4,b,b,5,b,7,6,b,8,9?  (I think that came out right; time to try it out.)

If you run the script you’ll simply get a one-line listing of page numbers. These can be pasted right into your browser’s “print” menu (if you’re lucky enough to be using GNOME).

And I should mention the other minor issue: a standard USA stapler has about a 4 inch (actually 9 cm) arm reach.  But you need 4¼ inches (obviously for 8½-inch wide paper).  So get creative or get an extended-arm stapler.

My vision is that the burgeoning class of new authors writing 100-page e-books can print-publish their own books basically for free.  Maybe it costs them a quarter to print each book, and two stamps to send it in the mail.  But for about $1 USD a new author can hit quite an audience without a publisher or bank account.  Yes, much of the world will just read e-books and manuals online, but there is still a broad group who doesn’t want to read more than a blog post-length article at their screen.  They can still benefit from printed copy.  I’m planning on devoting a whole shelf just for my own crude-bound “books”.  Next step is to find a clean way to have a quarter-inch flat binding that will take a label-maker sticker.

BTW, printer ink is a scam:http://www.csmonitor.com/2008/0109/p14s01-stct.html

O’Reilly’s Short Cuts are a great for printing.  And O’Reilly is also supporting ePub by releasing 400 commercial books for the format.

ALSO NEW: You can use FBReader or Calibre for desktop reading: apt-get install fbreader calibre

ALSO NEW: I also just bought a handful of Pragmatic Programmers books in e-format, as .epub, .mobi, and .pdf. Those are looking great on both my Android and desktop. Some PDFs are still worth printing.

It’s worth mentioning that some people hate syntax highlighting, and claim that it is distracting or gives them a headache.  My guess is that most of these people have never had the chance to work with a subtle-yet-attractive color scheme.  It’s a pretty well-accepted fact syntax highlighting is useful for catching typos and various bugs.  It takes a bit of training to notice when something highlights in an unexpected way, or doesn’t highlight when it should, but an eye for it can really improve your code quality.

Philosophy

• Nothing should be bold unless it’s very important and should stand out above all other things. Functions/classes are good for this.  (I’m actually using a font/size now that does not support bold.)
• Colors should be easy on eyes to stare at for many hours.  The background should be extremely comfortable for your eyes, so not white, and probably not even black.
• Todos should stand out just a little (underline looks nice).
• Inline comments should stand out less than anything else.
• Code “commented out” should be nearly invisible.
• But real in-code documentation should have strong emphasis and beauty.
• Everything should be differentiated, including identifiers, keywords, 3-4 types of comments, strings, types, constants, conditionals, control structures, and more.
• Strings should be a nice color since much documentation should exist in source code as strings (think “”" comments in python or /** in java).
• Normal text should be colored either black or white so it is obvious if something is not recognized syntax.
• No two used colors should be highly similar.  Beauty is secondary to contrast.
• Each language will have some special syntaxes that need specific attention.  E.g., py, pl, sh, html, vim, rst, moin, mail, etc.  It’s helpful to have a variety of file types hanging around and fire up your editor on each of them for the scheme you’re developing/modifying.
• The scheme should look nearly identical whether being used in an xterm or in a GUI.  E.g., looks the same in vim as gvim.
• Really needs to support 256-colors.  There are just too many different highlights to try and cover in 16 colors.

I’ve described the different types of comments in this related post.

Have a look at the repository of color schemes for vim.  There is also a nice screencapture of the repository.

Download my pythoncomments patch to enable python.vim syntax recognizer (for v2.6.3, maybe already part of your Vim installation) to make use of the various features of this article.  The patch is not necessary, but helps with some comment highlighting.

Different Types of Comments

Of course the code highlighting plugin this blog uses isn’t able to make much sense of these, but Vim can take it further with a little tweaking.

# look out for...
#print 'disabled
##print 'disabled'
for x in xs:  # iterate over xs
### King comment.
###print x
"""Multi-line chunk of commented-out code.  Sometimes only first sentence is significant."""
'''multi-line chunk of docstring'''
"string"
'string'
# TODO: add locking to...
######################################################################
#---------------------------------------------------------------------
# @param foo The metavar that never dies

I’ll break these down by line number.

1. is just a traditional comment that exists because something out of the ordinary is happening around the code.
2. a disabled line of code
3. another disabled line that is easier for a highlighter to treat.
4. an end-of-line comment; the most subtle form.
5. is a descriptive comment marking the purpose of a section of code.
6. a statement that only executes when in “debug mode”.  Perl’s “smart comments” are the only facility I’m aware of that uses this, but I think they’re pretty clever.
7. a formal piece of documentation, usually spanning multiple lines.
8. a simple way to comment out a large chunk of code.
9. just a string
10. same as 7; probably should not be differentiated, but some people prefer to the “dirk” version whenever possible.
11. a codetag (a controversial convention many use to refer to items that should (or do) have tracker tickets associated; described in PEP 350).
12. a break in major section of the file.
13. a minor section break.
14. a “javadoc”-style piece of documentational markup.  Also available with Epydoc.

View From Vim

For a more visual example, here’s a screenshot showing the best I can make Vim do today. Featuring the Adobe color theme, tokenized by python.vim.

It is certainly within reach to make Vim, emacs, and probably many other editors make use of these semantics in their syntax highlighting.  That color scheme (hacked up for this display) is not (yet) perfect, but it does the job of showing most of the differentiation. If there are so many purposes for the different comments, shouldn’t your editor make those differences obvious?

I’ll offer more details about getting this working, and the value of highly differentiated syntax, and what makes a good color scheme, in a near-future post.

I do most of my development work (bash, ipython, vim) in a terminal. Some of it involves working remotely on other machines, and demonstrating things for others (e.g., pair programming). I need different setups for each of these. I spend most of my time hacking away in a local shell, so for that I want optimized font, color, size, etc. Then when I share I need to make the font way bigger. And when I’m on a different machine I like to have a different color window (in addition to a custom prompt color) so I don’t forget where I am. So those are the three basic tweaks that are critical for most people. You can probably make those tweaks with most xterm-clones.

I’ll assume you’re using GNOME already. You can get a feel for the gnome-terminal feature set by simply firing up an instance: Applications -> Accessories -> Terminal. But I suggest you right-click it instead, and “Add this launcher to Panel“. In fact, do that a few times; we’ll customize each differently.

Now from an open instance, you can explore the options by invoking:

$ gnome-terminal –help

Profiles

There are profiles. Create a few. This is pretty important. If you’ve only got a “Default” profile, every time you change it, you’ll affect every other open window.

Font

I’m very picky about having the tiniest font that will work. I can actually look at 380 lines of code at a time on a humble (1920×1200) 15″ laptop screen. To achieve this I have to go with a non-antialiased (bitmap?) font. The generically named “Monospace-7″ is not beautiful, but works well for optimizing screen real estate. It’s also the only bitmap font I’ve seen that renders italic/oblique well.

Monospace is also the only font I’ve seen able to scale down to 6-point and render readably. Try it — pretty amazing! I can now look at 570 lines of code (if I cheat and narrow the last window a bit) in 15″! I’ve got great near vision, but I have to be really close to the screen to read it at 6, so I’ll go back to 7. You could also easily install other more appealing bitmap fonts like Terminus (apt-get install xfonts-terminus). In fact, terminus at 8-point fits the same number of characters as Monospace at 7-point. Too bad Terminus doesn’t do 6!

Remember to stop all running gnome-terminal instances to get a new font to show up in the list.

CLI Options

Customize the CLI options with right-click, Properties. Look at the Command field and start adding options there. The first I like to add is –geometry=80×50 to set the window size; the default of 80×25 is always too small. I like to start out with a meager 80 columns to make it obvious when I’m exceeding the good old rule.  Of course I often widen manually if I’m not in code.

Automatic Remote Login

My favorite trick is to automate logging in to remote machines. It’s as simple as adding another option: –execute ssh YOU@SOMEHOST