Scatterbrained: Revisiting the “doing” CLI

Back in 2011 I started tracking the minutiae of my work days using VoodooPad. It was partly for record keeping, but mostly to be able to walk away from my computer and still be able to remember what I was doing when I got back. I can be very, very scatterbrained. After VoodooPad, I had a system going using QuickQuestion and nvALT. Then it was Day One. By 2014 I’d come up with a solution in the form of a command line utility called doing. The journey was well documented up to that point in a series called, appropriately enough, “Scatterbrained.”

I haven’t written much about doing since then, but I continue to use it daily. It’s come a long way. It not only creates rich logs of my time at my computer, it also handles time tracking and reporting and integrates with my system via LaunchBar, various automations, and GeekTool. You know how git log can be really useful after a long night of hacking, or a few days of being away?¹ This is that, but for everything else, and it’s brimming with handy features.

Given I’ve been working on this as needed for 5 years now, it hasn’t felt like a descent into madness as much as a gentle slide into areas of questionable judgment. I find it a very useful tool, though, as long as I don’t think too much about how much time I’ve put into it.

After publishing a few new versions over the last couple of months, I thought it might be time to remind potential users (people who read this blog and enjoy my brand of madness) that it exists. This is going to be a longish post where I get to talk giddily about all the cool stuff this little tool can do. If you’re already sold — or at least enough to skip the spiel — and want to skip right to the documentation and the latest version, just head straight to the doing project page.

Before we go much further, I should make something clear. doing is primarily designed for use from the command line in your terminal of choice. It’s by a nerd, for nerds. I have, however, integrated it with LaunchBar and other systems over the years. It’s flexible. The LaunchBar action is included below, if you’re interested.

The basic idea of doing is to provide a way to keep track of the little things I work on over the course of a day, both as a reminder after I leave and forget what I was working on, and as a way to track my productivity.

It does this using a terminal command with an intuitive syntax. The command updates a simple TaskPaper-formatted text file. (You may be aware of my love of plain text…) It does some automatic time tracking, has a variety of tools for viewing your entries, and includes a few ways to do things like filtering, flagging, or archiving them. More tools than I’ll be able to fit into a blog post, even a long one, but the documentation is detailed and up-to-date.

If you’ve used Git before, the concept will be familiar. Just type in what you were working on, like you would for a commit, including as many notes as needed, and hit Enter to add it to the log. Doing even has query features that I dare say make it even more useful than git log. Sometimes. It depends on the situation. How about this: “in a very particular set of circumstances for a very specific set of needs, doing is more useful than git log.”

The Basics

You can install doing using Rubygems, i.e. gem install doing (you might need sudo gem install doing depending on your configuration). Once that’s successful, you can start recording entries right away, but I’d wait until you’ve read through the configuration section of the docs before going overboard. There are a few things you’ll want to adjust, including the location and name of your “doing file.”

Like git, doing uses subcommands, meaning you call doing and the next word in the command will determine what action it performs. This allows for (almost) natural syntaxes for most actions. The main command for adding a new entry is now, i.e. doing now [what you are doing]. now is also aliased as next as of recent versions, just because that sometimes makes more semantic sense. Like I said, almost human syntax.

You only need quotes around the entry if it contains characters the shell needs escaped, so I can just type doing now writing about doing. An entry will be created with the current time as the start time. There are features for modifying the start and finish times using natural language, but we’ll get to those in a bit.

You can add @tags in the text of an entry to allow for future sorting and filtering fun. (Tags use @ instead of #, first because that’s a TaskPaper standard, and second because # causes a lot of extra work on the command line.) You can also include notes with a -n "[Note Text]" flag.

If you want to modify tags and notes after the fact, just run doing tag writing to add @writing to the last entry, or doing note to open an editor, the results of which are appended as a note. And when you’re done doing what you’re doing, you can just run doing finish to add a stop time to the last task. Remember, all of this is stored in a TaskPaper file that you can manually edit at any time. Just run doing open to open the file in your default text editor.

You can see all of doing’s commands by running doing help. That will show you something like:

NAME
    doing - A CLI for a What Was I Doing system

SYNOPSIS
    doing [global options] command [command options] [arguments...]

VERSION
    1.0.25

GLOBAL OPTIONS
    -f, --doing_file=arg - Specify a different doing_file (default: none)
    --help               - Show this message
    --[no-]notes         - Output notes if included in the template (default: enabled)
    --stdout             - Send results report to STDOUT instead of STDERR
    --version            - Display the program version

COMMANDS
    add_section  - Add a new section to the "doing" file
    archive      - Move entries in between sections
    choose       - Select a section to display from a menu
    colors       - List available color variables for configuration templates and views
    config       - Edit the configuration file
    done         - Add a completed item with @done(date). No argument finishes last entry.
    finish       - Mark last X entries as @done
    grep, search - Search for entries
    help         - Shows a list of commands or help for one command
    last         - Show the last entry
    later        - Add an item to the Later section
    mark         - Mark last entry as highlighted
    meanwhile    - Finish any running @meanwhile tasks and optionally create a new one
    note         - Add a note to the last entry
    now, next    - Add an entry
    on           - List entries for a date
    open         - Open the "doing" file in an editor
    recent       - List recent entries
    sections     - List sections
    show         - List all entries
    tag          - Tag last entry
    templates    - Output HTML templates for customization
    today        - List entries from today
    undo         - Undo the last change to the doing_file
    view         - Display a user-created view
    views        - List available custom views
    yesterday    - List entries from yesterday

You can get help for any subcommand by running doing help [command], e.g. doing help meanwhile.

Seeing What You’re Doing

There are myriad ways to filter and view your tasks. Commands like doing recent, last, today, and yesterday are pretty self-explanatory, but the real workhorses are the doing show and doing view commands. Combined with custom templates, you can do everything from terminal time tracking to viewing HTML-formatted work logs in the browser. You can even get CSV and JSON output for passing to other tools.

doing show accepts a section name and/or tag names. By default doing creates sections (what would be projects in TaskPaper) for “Currently” and “Archive”. If you use doing later, it automatically adds a “Later” section. You can add as many custom sections as you want, either by editing the file directly (doing open) or by using doing add_section [Section Name].

For example, I keep a running list of random ideas in a separate section (called “Ideas,” obviously). It lets me get ideas out of my head without going down rabbit holes or worrying that I’ll forget about them. I can add to that section using doing done -s Ideas My great @idea. Note that I used done because I’m not time tracking these, so it makes sense to add them as finished entries right away, and I also included an @idea tag. I actually have an alias for this that lets me just type idea My great idea, and the word “idea” is always automatically tagged using a tagging whitelist in my configuration file. (See the section on autotagging further down.)

Now I can just run doing show Ideas and see all of my ideas listed out. I can also run doing show @idea to show entries tagged with @idea across all of my sections. And I can combine the two to show just entries with a particular tag within a particular section, e.g. doing show Ideas @coding. You can even include multiple tags and use the -b [AND|OR|NONE] flag to define the boolean used to search. So doing show -b OR @idea @brainstorm @thinking would show any entry tagged with any of @idea, @brainstorm, or @thinking.

The view command is similar to show, but allows you to customize all of the aspects of the view, from what section/tags to include to the output template. Using the .doingrc config file (documentation), you can define named views with custom parameters and templates. I have a view set up called “ideas” that lets me run doing view ideas and have my ideas shown like this:

This is accomplished with a section in the YAML-formatted config file:

views:
  ideas:
    template: "\U0001F4A1 %softpurple%title %boldblack%note%default"
    order: asc
    count: 20
    section: Ideas
    wrap_width: 0

Among my other views is one for GeekTool² which puts my three most recent entries across all sections on my Desktop, formatted as single lines with no notes, and a date format string that keeps everything aligned nicely (when used with a mono font).

The view for that one:

views:
	geektool:
	    date_format: "%a %_I:%M%P"
	    tags_bool: NONE
	    tags_color: green
	    template: "%magenta%date %boldcyan> %white%title%default"
	    order: asc
	    count: 3
	    section: All
	    tags: cancelled
	    wrap_width: 0

There are a ton of options for the view and show commands, so run doing help show and doing help view to get a feel for them. There’s even an HTML output option (doing show [section/tag] -o html) you can pipe to bcat or hcat. It works with any view or show command, just add -o html anywhere after the subcommand, e.g. doing show Archive @marked -o html|bcat. You can also output any show or view command as JSON or CSV, overriding the template to provide structured data.

Tracking Time

At the outset I had no intention of using doing for time tracking. Time tracking kind of evolved over time. It’s still not going to beat an actual time tracking app, nor any automated time tracking like Timing. Nonetheless, doing has some handy tools.

All of these commands operate on the “Currently” section by default, and can be pointed at other sections using -s SECTION_NAME. The section name is fuzzy matched and case insensitive.

As I mentioned previously, commands like now and done allow various combinations of --took and --back for fudging times if you haven’t been religiously logging every step. Using --back lets you modify the start date with natural language, e.g. doing now --back 30m Starting the thing to start it 30 minutes ago. Similarly, doing done --took 1h will finish the last task by adding an hour to whatever the start date was. You can also add a task complete with retrospective time tracking using a single command, e.g. doing done --took 2h working on website typography.

There’s also a meanwhile command that lets you maintain an overarching task, like a major project, while you knock off smaller items within it. You can start a meanwhile task with doing meanwhile Working on the @ACME job, optionally with a --back 2h, and then next time you run doing meanwhile it will finish that task, even if it wasn’t the last thing added. Starting a meanwhile task always finishes any currently-running “meanwhile”.

You can finish your last task with doing done (with no entry text), or doing finish, both of which accept both --back and --took. Supply a count with finish to finish the last X tasks, i.e. doing finish 5. One useful feature of finish is the --auto flag which will go through the last X entries and add finish times to each one based on the start time of the next one. It comes in handy.

Similar to the doing finish --auto trick, if you use -f (--finish_last) with the now command, it will check the previous item and see if it’s been tagged @done yet. If not, it will finish the previous entry with a timestamp matching the start time of the new item. When you’re in a mode where you’re logging each task as you start it, rather than retroactively (as I often do), this feature just allows you to skip the step of completing a task before starting a new one.

All of the view commands can show how long tasks took. In the default views, any task that has an interval between its start and end times will show a HH:MM:SS token at the end of the task. This can be modified in the various templates. Running a view or show command with --totals will add up total times by tag, and you can use --only_timed with the show command to only output entries with elapsed times.

Tagging and Time Tracking

Total times are displayed by tag. Adding a project tag allows you to see how much time went into each project. You can have multiple tags, of course, so you can have three different project tags but all of them get a @coding tag, so you can see each project but also a total for time spent in the coding context.

When it comes to tagging projects for time tracking, there are a couple of approaches. First, you can just manually add a tag to denote what the time should be attributed to, and you can overlap tags as needed. But you can automate this by using a “local” .doingrc file in your project directory. Configuration files found anywhere between the current directory and the root are applied on top of the main configuration, cascading such that the closest one takes precedence. Only the keys you want to override are needed.

For example, in the directory for Marked development (~/Code/Marked) there’s a .doingrc file that just contains:

---
default_tags: [marked]

And one in the parent directory (~/Code) with:

---
default_tags: [coding]

Any entry I make from the Marked directory (or any of its subdirectories) will get tagged “@marked @coding” automatically.

You can also tag the last task retroactively using doing tag TAGNAME. Or just run doing open and edit the file directly.

Autotagging

Autotagging in doing consists of “whitelisting” and “synonyms”. The synonyms feature is pretty cool: you can add a tag and then list all of the keywords that should trigger that tag. If a keyword appears in an entry, that tag gets added automatically.

The whitelist, on the other hand, is a list of words that should be directly turned into tags. If the list contains “design” and I type “Working on site design,” that gets converted to “Working on site @design.” Whereas if I had “design” set up with synonyms which include “typography,” I could write “Working on site typography” and it would become “Working on site typography @design.” Handy.

Automatic tags will never be repeated within an entry, so if you manually tag any of the defined tags it will skip adding them, and only the first occurrence of a keyword is converted into a tag.

Time Reporting

You can include time intervals in any template or view using the %interval template key. If a project has no time between the start and finish timestamp then nothing will display. This is great for seeing how much time you’ve put into things over the day.

To get a real report you can use doing show or doing view with --only_timed and --totals. With the show command you can use --from to specify a date or date range, and then provide @tags to show only the time spent on a specific project within a specific range.

Any project or activity you have a tag for is easily reported. An example command would look like:

doing show -t --totals --from="saturday to sunday" @writing

This gives me a report of what I’ve spent my time writing about this weekend. In this case, it’s all about this post. It will show all of the pertinent entries, with a block at the end containing totals for each tag and total elapsed time. Here’s what the above results look like for the weekend:

--- Tag Totals ---
doing:   00:02:15
writing: 00:02:15

Total tracked: 00:02:15

The more tags you have, the more detailed and flexible the reporting results are. And, as with any of the views, you can output this to nicely formatted HTML by adding -o html to the command.

By having overarching project tags as well as tags for various contexts like @designing, @coding, @writing, it’s easy to see how much total time went into a project, as well as exactly where the time was allocated. You can tag for anything. And if you make good use of the autotagging features, most of the work is done automatically.

Side note, when I forget to add a doing entry but wished I had, I load up Timing.app and am able to see what files I had open, when, and for how long. It’s the other half of my “too-scatterbrained-for-time-tracking” time tracking system.

Even More Natural Language

I’ve created a few aliases and functions as doing shortcuts over the years, but it’s all pretty simple. A verb with a function that specifies a section or adds tags, etc. Something as simple as alias thinking="doing now -s Ideas Thinking " means I can type thinking about digging back into React again and have that added to my Ideas list.

You can also use a wrapper function to handle all kinds of natural language. Here’s an example for Bash:

# Allows commands like `im coding bash scripts for doing`
im() {
	local verb=$1

	shift

	# special handling for `im thinking ...`
	if [[ $verb == "thinking" ]]; then
		# trim about
		[[ $1 == "about" ]] && shift
		# Add to Ideas with @idea tag
		doing done -s Ideas "$* @idea"
	# Special handling for `im coding ...`
	elif [[ $verb == coding ]]; then
		# Add to Projects with @coding tag
		doing now -s Projects "$* @coding"
	# Any other "ing" gets the verb tagged
	elif [[ $verb =~ ing$ ]]; then
		doing now "@$verb $*"
	# `im done ...` or `im finished ...`
	elif [[ $verb =~ (done|finished) ]]; then
		# If the first word after done is a present-tense
		# verb, tag it
		if [[ $1 =~ ing$ ]]; then
			doing done "@$*"
		else
			doing done "$*"
		fi
	# Fine, be boring
	else
		doing now "$*"
	fi
}

Working with entries

The beauty of the whole system is that you can open the doing file in your text editor or in TaskPaper at any time to make quick work of modifying your entries. You can specify an editor app in your .doingrc and then run doing open at any time to open the file.

In an effort to make things simpler, though, doing includes commands for flagging entries, archiving them, and batch moving them between sections. Want to archive everything in your Ideas section, keeping only the last three? doing archive --keep=3 Ideas. The archive command also accepts --to=[Section] to move entries to a section other than Archive. You can also specify a @tag to archive or move entries matching a tag, rather than just the last entries chronologically.

Automating

I won’t dig deep on automating, but I’ll point out that I’ve had great results using git hooks to update doing. You’ll also find scripts for LaunchBar and Alfred, as well as Bash, Fish, and Zsh completion on the project page. Whether it’s GeekTool, browser pipes with bcat, LaunchBar, or just the terminal, doing is designed to work with other tools as much as possible.

Phew

That got long, and it’s just an overview — I didn’t even cover all of the features. Or this is just the rantings of a madman. Hard to say sometimes. As mentioned previously, the documentation is complete and the built-in help system is verbose enough to fill in any gaps.

Like Marked 2, SearchLink, Reiki, na, or any of my pet projects, doing is one of my babies. I use it every day, and I bend it to fit my needs as they arise. I don’t need a large user base or an influx of donations to make it worthwhile, it’s fun to work on and I find it consistently useful. That said, feel free to try it out!