Shell Tricks: convert file urls to UNIX paths

Today’s shell trick is for converting file:// urls into valid shell paths. This isn’t a terribly common scenario, but I occasionally work with tools, especially in GUI applications, that output file urls and need to change something like:

file:///Users/ttscoff/Desktop/my%20file.txt

into:

~/Desktop/my\ file.txt 

These are Bash-specific, due mostly to some variable mangling syntax, but could easily be converted for zsh and others.

I’ve broken functions for unescaping percent-encoded input, shell escaping regular text, and trimming full paths to tilde abbreviations (when needed) into separate utility functions because they’re reusable and useful in other functions and aliases.

I handle shell escaping with the default Ruby Shellwords module. It’s fast and covers edge cases, avoiding a lot of sed/awk work. There are modules in other scripting languages as well, but Ruby and the Shellwords module are standard on all OS X systems (and what I know best). Substitute based on your personal preference.

# Ruby ShellWords escape
shellesc() {
    local output
    # If any arguments are passed to the function, assume that's the input
    if [[ $# == 0 ]]; then
        output=$(ruby -e 'require "shellwords"; puts Shellwords.escape(STDIN.read.strip)')
    # otherwise, take input from STDIN so it can be used in piped commands
    else
        output=$(ruby -e 'require "shellwords"; puts Shellwords.escape(ARGV.join(" ").strip)' $@)
    fi

    echo "$output"
}

I also use Ruby (CGI class methods) to unescape urls when working in the shell. It will convert any percent-encoded entities to their natural state:

# Ruby cgi unescape
unesc() {
    local output

    if [[ $# == 0 ]]; then
        output=$(ruby -r cgi -e 'require "cgi"; puts CGI.unescape(STDIN.read)')
    else
        output=$(ruby -e 'require "cgi"; puts CGI.unescape(ARGV.join(" "))' $@)
    fi

    echo "$output"
}

Lastly, a quick utility using sed to replace full paths in home directory with a tilde (~), e.g. /Users/ttscoff/Desktop becomes ~/Desktop. It’s more readable and allows more portability as the tilde will expand to whatever the current user’s home folder is.

Note: When input is recieved from STDIN instead of arguments, it automatically calls the shell escape function to avoid losing existing escaping through the read command. Thus a call to shellesc | shorthome in other functions is redundant (though not fatal).

shorthome() {
    local input
    if [[ $# == 0 ]]; then
        read input
        input=$(shellesc "$input")
    else
        input="$@"
    fi
    echo -n "$input" | sed -E "s/^${HOME//\//\\/}/~/"
}

Lastly, here’s the function that combines the previous utility functions to convert a file:// url to a shell-escaped path.

If you pass -c as the first argument (or only argument if you want to use piped input from STDIN), results are copied to the clipboard (using pbcopy, which is OS X-only). The bulk of the function is a simple piped chain of the above functions, with a quick variable mangling in Bash to remove the file:// prefix.

# convert a file:// URL to shell path
url2path() {
    local input output
    local copy=false
    if [[ $1 == '-c' ]]; then
        copy=true
        shift
    fi
    if [[ $# == 0 ]]; then
        read input
    else
        input=$@
    fi

    # 1. Replace 'file://', `%20` (space), and other entities in the url
    # 2. Add shell escaping for spaces and any non-legal characters
    # 3. Replace hardcoded home paths with the tilde abbreviation
    output=$(unesc ${input#file:\/\/} | shorthome)

    if $copy; then
        echo -n "$output"|pbcopy
        echo "Result in clipboard"
    else
        echo -n "$output"
    fi
}

With that code sourced in your ~/.bash_profile, you can use commands such as echo "file:///Users/ttscoff/Desktop/my%20file.md" | url2path or url2path file:///Users/ttscoff/Desktop/my%20file.md to get the filepath, either directly or as part of another script.

The command (usable as an alias) pbpaste | url2path -c will convert a url in the clipboard into a file path, in place, ready for pasting.

Ryan Irelan has produced a series of shell trick videos based on BrettTerpstra.com posts. Readers can get 10% off using the coupon code TERPSTRA.

Brett Terpstra

Brett is a writer and developer living in Minnesota, USA. You can follow him as ttscoff on Twitter, GitHub, and Mastodon. Keep up with this blog by subscribing in your favorite news reader.

This content is supported by readers like you.