Welcome to the lab.

Shell Tricks: Quick line numbering

[Tweet : ADN : nvALT]

It’s often useful to quickly see line numbers when viewing source code files. The less command and related tools can display with line numbers. There’s also the nl command, but it doesn’t number blank lines and tends to mess up formatting. There’s are ways that are more flexible and better looking. Enter grep and paste.

grep version

If you pass a “match everything” pattern to grep (.*) and include the -n flag, you’ll get the entire file passed back with line numbers followed by colons.

A quick alias makes this easily accessible:

alias grepno="grep --color=never -n -E '.*'"

Now you can just type grepno [filename.txt] to see numbered output.

$ grepno Rakefile
46:Rake.application.options.trace = false
48:run_time = Time.now

The nice thing about the -n switch is that you can actually search with it and see the resulting line numbers for just the matching lines. Same with ack, ag (Silver Searcher), and others.

I don’t love the output, though.

paste version

The paste command has a somewhat esoteric function:

The paste utility concatenates the corresponding lines of the given input files, replacing all but the last file’s newline characters with a single tab character, and writes the resulting lines to standard output.

For this tip, we’re overriding the newline-to-tab conversion (with the -d flag) and replacing newlines with TAB-newline, maintaining the newline but indenting it. sed = will number the lines in the output, and the -s switch on paste will restore our whitespace keeping the line number justified to the left.

# output a text file with line numbers
lno() {
    if [ $# == 0 ]; then
        echo "No filename provided."
        sed = "$1" | paste -s -d '\t\n' - -

This will give you nicely formatted, line-numbered output.

$ lno Rakefile
46  Rake.application.options.trace = false
47  verbose(false)
48  run_time = Time.now

You can pass it to less for (in my opinion) better-looking output than less’s default -n switch. You can also pipe it to grep to filter, generating the same paginated output as grep -n with better formatting. You can also redirect to a file (> output.txt) or the clipboard (| pbcopy) for line-numbered text you can share.

Web Excursions for November 23, 2015

[Tweet : ADN : nvALT]

slalert!: News Alerts in Slack
This new tool from Diffbot (I’m a fan of Diffbot for all kinds of web scraping) allows you to scour the web for mentions of your app or project and get alerts in Slack whenever it’s mentioned. Nifty.
Stock Up
Another great source for free stock photos, sourcing and searching 26 different free photo sites.
Rover – simple file browser for the terminal
A small, terminal-based (curses) utility for navigating file systems. vi-like keybindings, integration with environment variables, and file management capabilities.
Using Swift To Make Command Line Scripts - Part 1
A great rundown on using Swift to create CLIs with argument handling and full functionality.
A set of tools to assist in creating command line applications and tools. Colorize output and provide user interaction (ask, choose, agree).

Black Friday Deal at Udemy: 17,000+ online courses for $10 (48 hours only)

[Tweet : ADN : nvALT]

Black Friday $10 728x90

I’ve partnered with global online learning marketplace Udemy to bring you a Black Friday $10 deal on 17,000+ of their best selling courses.

Learning is no longer limited to a classroom or a book. We live in a world where we can learn new skills and concepts on-demand, at our own pace, and on an amazing array of devices. Udemy has both master and mini courses on everything from programming to photography, and I encourage you to take a look.

I think most of my readers will gravitate to the wide array of programming courses (covering everything from web development, JavaScript, and advanced libraries like Angular JS, to iOS, Android, Mac, and Windows programming), but who knows? You may find a new passion (like writing or yoga), master new business skills (like Excel), or maybe even a new career.

Starting today, you can get courses for $10, which is up to 98% off. The $10 deal only lasts 48 hours. After Thursday the price starts rising until Black Friday, when the sale ends at a price of $15 for courses. If you’re looking to learn, stock up early!

Smarter keyboard shortcuts for Finder

[Tweet : ADN : nvALT]

I frequently mention the capability of OS X to assign keyboard shortcuts to Services and other functions. At its most basic, though, the keyboard shortcut functionality can enhance and customize any application. As an example, Finder can be extended nicely for the keyboard-inclined.

Most features in Finder already have shortcuts, as you’ll see if you pull down any menu from the menu bar. There are a few that don’t, primarily contextual menu items, that can have shortcuts assigned to speed up your workflow.

To assign shortcuts, open System Preferences and navigate to the Keyboard pane. Once there, choose the Shortcuts tab and select App Shortcuts from the sidebar. Use the “+” button to add a new one. You can select “All Applications” for universal shortcuts, or a specific app (in this case Finder). Type the title of any menu item—exactly, including upper and lower case—and assign a shortcut. That’s it.

The most useful shortcuts I have assigned:

Show Package Contents
I use ⌃⌘P for this. When a bundle file (e.g. application or document bundle) is selected, pressing the combination will open it as a folder in Finder, the same as ⌘↓ does when a folder is selected.
Note that an ellipsis is not three periods, it’s a single character created by ⌥; (Option-Semicolon). With this one defined, pressing the key combo will pop up a dialog on any selected file(s) that I can use to tag quickly in Finder.
I had assumed for a long time that—because Finder changed the menu item title to “Compress [filename]” in the contextual menu—you couldn’t easily assign a shortcut to it. I mentioned on Mac Power Users that I’d used BetterTouchTool to solve the issue. While the BTT trick is very cool, it turns out (thanks @macfixer) that you can just use “Compress” as the menu item title and the shortcut will work.
Sorting and Display shortcuts
There are default shortcuts for sorting (Name, Modified date, etc.) and display grouping, but I’ve always found them obtuse. I assign these to standard Function keys, e.g. F1-F3 handle quickly sorting by Name, Date, or Size on my system.

Shell Tricks: convert file urls to UNIX paths

[Tweet : ADN : nvALT]

Today’s shell trick is for converting file:// urls into valid shell paths. This isn’t a terribly common scenario, but I occasionally work with tools, especially in GUI applications, that output file urls and need to change something like:



~/Desktop/my\ file.txt 

These are Bash-specific, due mostly to some variable mangling syntax, but could easily be converted for zsh and others.

I’ve broken functions for unescaping percent-encoded input, shell escaping regular text, and trimming full paths to tilde abbreviations (when needed) into separate utility functions because they’re reusable and useful in other functions and aliases.

I handle shell escaping with the default Ruby Shellwords module. It’s fast and covers edge cases, avoiding a lot of sed/awk work. There are modules in other scripting languages as well, but Ruby and the Shellwords module are standard on all OS X systems (and what I know best). Substitute based on your personal preference.

# Ruby ShellWords escape
shellesc() {
    local output
    # If any arguments are passed to the function, assume that's the input
    if [[ $# == 0 ]]; then
        output=$(ruby -e 'require "shellwords"; puts Shellwords.escape(STDIN.read.strip)')
    # otherwise, take input from STDIN so it can be used in piped commands
        output=$(ruby -e 'require "shellwords"; puts Shellwords.escape(ARGV.join(" ").strip)' $@)

    echo "$output"

I also use Ruby (CGI class methods) to unescape urls when working in the shell. It will convert any percent-encoded entities to their natural state:

# Ruby cgi unescape
unesc() {
    local output

    if [[ $# == 0 ]]; then
        output=$(ruby -r cgi -e 'require "cgi"; puts CGI.unescape(STDIN.read)')
        output=$(ruby -e 'require "cgi"; puts CGI.unescape(ARGV.join(" "))' $@)
    echo "$output"

Lastly, a quick utility using sed to replace full paths in home directory with a tilde (~), e.g. /Users/ttscoff/Desktop becomes ~/Desktop. It’s more readable and allows more portability as the tilde will expand to whatever the current user’s home folder is.

Note: When input is recieved from STDIN instead of arguments, it automatically calls the shell escape function to avoid losing existing escaping through the read command. Thus a call to shellesc | shorthome in other functions is redundant (though not fatal).

shorthome() {
    local input
    if [[ $# == 0 ]]; then
        read input
        input=$(shellesc "$input")
    echo -n "$input" | sed -E "s/^${HOME//\//\\/}/~/"

Lastly, here’s the function that combines the previous utility functions to convert a file:// url to a shell-escaped path.

If you pass -c as the first argument (or only argument if you want to use piped input from STDIN), results are copied to the clipboard (using pbcopy, which is OS X-only). The bulk of the function is a simple piped chain of the above functions, with a quick variable mangling in Bash to remove the file:// prefix.

# convert a file:// URL to shell path
url2path() {
    local input output
    local copy=false
    if [[ $1 == '-c' ]]; then
    if [[ $# == 0 ]]; then
        read input

    # 1. Replace 'file://', `%20` (space), and other entities in the url
    # 2. Add shell escaping for spaces and any non-legal characters
    # 3. Replace hardcoded home paths with the tilde abbreviation
    output=$(unesc ${input#file:\/\/} | shorthome)

    if $copy; then
        echo -n "$output"|pbcopy
        echo "Result in clipboard"
        echo -n "$output"

With that code sourced in your ~/.bash_profile, you can use commands such as echo "file:///Users/ttscoff/Desktop/my%20file.md" | url2path or url2path file:///Users/ttscoff/Desktop/my%20file.md to get the filepath, either directly or as part of another script.

The command (usable as an alias) pbpaste | url2path -c will convert a url in the clipboard into a file path, in place, ready for pasting.

A fix for slippery iPhones

[Tweet : ADN : nvALT]

It was on Systematic #133 with Rich Stevens where I was first introduced to Suction Cup Tape. I ordered a sheet before the podcast recording had ended. It was my own “pick of the week” soon after.

It’s a tape with traditional adhesive on one side, and a “micro suction cup” layer on the other side. You stick the former side to something such as your iPhone, where it affixes permanently (though the residue is easy to remove), and then that object can adhere to any flat surface, be easily removed, and re-adhered again (and again). When it stops sticking, just wipe it off with a bit of soap and water and it’s good to go.

I fell in love with the stuff, especially with my slippery iPhone 6+. Even when not fully adhered to something, it provided a non-slip surface that prevented dropping my phone or having it slide off of my dashboard, car seat, or a table jostled by a furry family member.

The problem was that it’s expensive to produce, and thus even more expensive to purchase and use. An 8.5x11” sheet runs around $15. Granted, the first sheet I bought has lasted almost a year across three devices, but it does eventually begin to wear out. I’d assumed that prohibitive costs would prevent it from ever being mass-marketed, but LizzyTape proved me wrong.

iPhone 6+ with LizzyTape
iPhone 6+ with LizzyTape (yes, it’s askew, and no, you can’t get a MinimalMac t-shirt anymore).

The LizzyTape Cellphone Sticker Mount is available on Amazon for $5.99. It’s a smaller patch of suction cup tape, and the price comes out about the same per square inch, but you can buy it as a ready-to-use version at a lower price than buying full sheets.

Assuming it lasts as long as the similar material I’m used to, one sticker on the back of a phone should be good for at least a year. I received a couple of them that I’d ordered this week, and it’s just as good as the tape I’d been cutting previously.

I can stick my phone to the bathroom mirror while brushing my teeth, my dashboard (or the empty carseat next to me because no one ever rides in my car—except for Emma, occasionally) while I’m driving, or just let it rest on my palm while walking for slip-free usage without a rubberized case or any actual adhesive.

I’d always cut my strips to full width of the device, and it was the edges that first started wearing out and peeling. The smaller format of the LizzyTape version makes sense, and will probably prove to be a better solution than my previous efforts.

I don’t generally review $6 products I found randomly on Amazon, but I’m excited about this one. If you’ve ever wished your phone, tablet, or small child would stay in place1, I recommend checking it out.

  1. Disclaimer: I do not have children, and I do not know how they work.

Curio 10 and Curiota

[Tweet : ADN : nvALT]

I consider Curio to be the most expansive brainstorming application on the market. With mind maps, lists, file links, notes, full project organization tools, and much more, it’s a complete productivity app. Version 10 just came out, along with a menu bar app for quick entry, and it continues to extend the capabilities of this powerhouse application.

I wrote a longer piece about Curio years ago for TUAW (now archived at Engadget), so if you’d like a foundation explanation of Curio, check that out. I also covered the release of version 9 on this blog, so I’ll go ahead and jump to the changes in version 10.

“Refinements” would be an understatement for many of the new features. While not adding complexity, Curio has added several features that better accommodate large collections of information. The most useful of these for me is the new “Stacks” feature.

Whereas previously you would be best off organizing your notes and information on a flat surface, stacks allow you to collect notes into piles and easily navigate, arrange, and sort the information.

Mind maps in Curio can now have automatically-sorted branches, as can list collections. Importing from iMindMap and MindNode is also improved. I’ve always found importing brainstorming sessions into Curio to be a delight—just drag and drop most file types or paste OPML, Markdown, text, and more, and have them converted to native formats in Curio—and these further enhancements only improve the seamless integration with my other tools.

The Library in Curio has always been an integral piece of its functionality, allowing shared and reusable resources, integration with Evernote, and additional project management capabilities. The new “Local” library shelf—with custom watch folders, fast searches and integration with Curiota (see below)—harnesses Spotlight to make working with information stored anywhere on your system an easy task.

The menu bar companion, Curiota, is a very handy addition. It lets you add notes on the fly from any app, but also lets you drag files and integrates with System Services, the OS X share button, and PDF services. It’s an omnipresent way to gather resources and notes for use within Curio.

I’m planning to write a bit more about the release for MacStories, but in the meantime you can check out the descriptions of all the new features on the Zengobi website.

Curio 10 is available for $129.99 (educational discounts available), and existing users can upgrade for $49.99. A two-week free trial of the full application is available on the Zengobi site.

A personal podcast feed for your audiobooks

[Tweet : ADN : nvALT]

I have a guest post from Andreas Zeitler for you today. Andreas is one of the hosts of the German podcast Der Übercast, and I’m happy to share this handy trick for liberating your audiobooks.

Podcasts are becoming an increasingly popular format on the Internet. One of my personal favorites is Brett’s Systematic (and, of course, Overtired with Christina Warren). While I like podcasts, I also listen to audiobooks.

Audiobook platforms, such as Audible, distribute a relatively historic audio format. Audible calls it AAX or AAX+, indicating it’s an AAC format. Ever since AAC has become the norm on the iTunes Store, the format has seen vast improvements, but audiobook haven’t. What if I told you that an audiobook could take up about 1/10 of its current space on your phone? That means instead of using 400MB, a more modern format would use 60MB.

That’s just one advantage though. Compared to podcast players, the audiobook apps have not seen many improvements in terms of player capabilities. Overcast for example has an intelligent way to cut out unnecessary pauses which reduces listening time significantly, without changing the overall speed. This means that if your audiobook seller would allow you to use a modern player, you could not only save time, but you’d also be free to choose which format you like best.

As a podcaster and media professional myself, I had an epiphany one day. Why not make my own audiobook podcast?

Bonus Tip: To be honest, this whole approach was inspired by Librivox. They offer an iTunes and RSS feed option for their (free) audiobooks. That’s quite cool if you want to listen to, say, The Tao Teh Ching.

Audio Files

The first thing we have to take care of is converting the audiobooks into a new format. I won’t offer any specific recommendations here, but there are several apps that allow you to record audiobooks while they are played, and then convert them (see Audio Hijack and AudioBook Converter). They’re inexpensive and available for many platforms – even iTunes itself is able to play Audible books back.

Once they are recorded, you can convert them into a new format (if your recorder app does not directly record into the desired format). As the time of this writing I personally prefer MPEG-4 Audio High-Efficiency or even HE AAC v2 (High-Efficiency v2). The file sizes this format produces are really small, with a higher quality at lower rates than older formats. If your podcast app doesn’t play these files back, you can choose a different, more common, format such as MP3 or a normal AAC. We’ll get to that shortly.

On a Mac I love Amadeus Pro by Hairersoft, because it has a batch processor to convert a bunch of audio files in one fell swoop. It is also possible to apply some processing on conversion such as an EQ, or compressor. Most audiobooks I listen to don’t come with music. One voice coming from one microphone means the format is mono. In case the original recording is stereo, you can save even more space by converting it to a mono file.

As an example I’ve used Michael Gerber’s The E-Myth Enterprise. The original recording was 483.3MB. Converting to AAC brings it down to 67.5MB.

Hints: If the audio file won’t play on the device, it could be that the app doesn’t support it, or the podcast app doesn’t reload your podcast XML properly after an update. Please talk with the developers if you experience issues. I was having problems with Overcast, Downcast was fine.

Podcast Creation

The creation of a podcast is also relatively simple. If you want, you can even write it yourself! (Though, I wouldn’t recommend it.)

A popular content management app is WordPress. It comes with a plugin architecture. Search for a plugin that allows to add audio files to blog posts (podcast plugins) and you’re good to go.

On a Mac you can use Feeder. This app allows you to create podcast feeds. It has a graphical interface like WordPress, and it handles all the uploading for you, too.

Bonus: Steve Harris, the developer of Feeder, has provided BrettTerpstra.com readers with a 20% off discount code for the direct purchase. Use ZEITLER20 at checkout!

Alternatively, you can use something like JustCast (for Dropbox) or Google Drive to host the podcast.

It is also possible to create a podcast feed of files from a folder, e.g. with DirCaster. This is probably the best method if you own a server.

Within a very short amount of time you will have the files on your device and you can start to listen to them at 2.1x cutting out all the unnecessary silence, giving you more free time while maintaining all information. Plus… you can brag about your nerdy skills in front of friends and family.

Andreas is 33.333% of Der Übercast, a 100% true German podcast. He’s also teaching you how to podcast at Podcast-Academy. For videos for your business, check out zCasting 3000.