Welcome to the lab.

Shell Tricks: shorten every line of output

[Tweet : ADN : nvALT]

Updated: Dr. Drang pointed out that the original functions were mostly working but flawed. I’ve updated this whole post.

This is a simple bash function that will take each line of the input piped to it and truncate it at a given length (default 70 characters), optionally inserting an ellipsis or other string if the line is truncated.

Here’s the basic trick, using sed with an example line length of 60:

short () {
	cat | sed -E "s/(.{60}).*$/\1/"

Here’s a more complete version of the function. It accepts a -l switch to truncate from the left instead of the right (default), and a -s STRING flag to allow the user to specify an ellipsis or other string to add to lines that have been truncated.

# Truncate each line of the input to X characters
# flag -s STRING (optional): add STRING when truncated
# switch -l (optional): truncate from left instead of right
# param 1: (optional, default 70) length to truncate to
shorten () {
	local helpstring="Truncate each line of the input to X characters\n\t-l              Shorten from left side\n\t-s STRING         replace truncated characters with STRING\n\n\t$ ls | shorten -s ... 15"
	local ellip="" left=false
	while getopts "hls:" opt; do
		case $opt in
			l) left=true ;;
			s) ellip=$OPTARG ;;
			h) echo -e $helpstring; return;;
			*) return 1;;
	shift $((OPTIND-1))

	if $left; then
		cat | sed -E "s/.*(.{${1-70}})$/${ellip}\1/"
		cat | sed -E "s/(.{${1-70}}).*$/\1${ellip}/"

These functions can be added to any file that’s sourced during login, such as ~/.bash_profile. Then they can be used like:

cat filename.txt | shorten 20

You can shorten the output of any command:

ls -1 | shorten 15

Example output without shorten:

$ ls -1 2016-04*

And with shorten to 10 characters, with ellipses:

$ ls -1 2016-04* | shorten -s ... 10

Shell Tricks: list files with most text matches

[Tweet : ADN : nvALT]

Here’s a Bash function for searching all text files in the current directory for a pattern, then listing the files containing matches in ascending order by number of matches. It’s mostly a proof of concept, but a useful companion to a basic grep search.

The meat of the script happens in an array declaration. It first uses grep -lIi -E "$patt" * 2> /dev/null to list files containing the provided pattern (case insensitive), ignoring binary files. The error redirect at the end of the command will ignore the errors thrown by directories. The results of this are fed to another grep command: grep -Hi -c -E "$patt" which outputs the match count for each file. The results are saved to the array.

After including the function in a sourced file (e.g. ~/.bash_profile), running matches -h will show the available flags and switches:

$ matches -h
Find files in the current directory containing the most occurrences 
of a pattern
	-c Include occurrence counts in output
	-r Reverse sort order (default ascending)
	-m COUNT Minimum number of matches required
	-h Display this help screen

	# search for files containing at least 3 occurrences
	# of the word "jekyll", display filenames with counts

	$ matches -c -m 3 jekyll

Here’s the function for pasting into ~/.bash_profile (or other sourced file):

# Find files in the current directory containing the most occurrences of a pattern
# switch -c: turn on display of occurrence counts
# switch -r: reverse sort order (default ascending)
# flag -m COUNT: minimum number of occurrences required to include file in results
# param 1: (required) search pattern (regex allowed, case insensitive)
# Results are output in ascending order by occurrence count
matches () {

	local counts=false minmatches=1 patt width=1 reverse=""
	local helpstring="Find files in the current directory containing the most occurrences of a pattern\n\t-c         Include occurrence counts in output\n\t-r         Reverse sort order (default ascending)\n\t-m COUNT   Minimum number of matches required\n\t-h         Display this help screen\n\n	Example:\n\t# search for files containing at least 3 occurrences\n\t# of the word \"jekyll\", display filenames with counts\n\n\t$ matches -c -m 3 jekyll"

	while getopts "crm:h" opt; do
		case $opt in
			c) counts=true ;;
			r) reverse="r" ;;
			m) minmatches=$OPTARG ;;
			h) echo -e $helpstring; return;;
			*) return 1;;
	shift $((OPTIND-1))

	if [ $# -ne 1 ]; then
		echo -e $helpstring
		return 1

	patt=$1; shift


	declare -a matches=$(while read -r line; do \
	                grep -Hi -c -E "$patt" "$line"; \
	              done < <(grep -lIi -E "$patt" * 2> /dev/null) \
	              | sort -t: -${reverse}n -k 2)
	width=$(echo -n ${matches[0]##*:}|wc -c|tr -d ' ')

	for mtch in ${matches[@]}; do
		if [ ${mtch##*:} -ge $minmatches ]; then
			if $counts; then
				printf "%${width}d: %s\n" ${mtch##*:} "${mtch%:*}"
				echo "${mtch%:*}"


Web Excursions for April 20, 2016

[Tweet : ADN : nvALT]

HoudahGeo 5 — Photo Geocoding & Geotagging for Mac
HoudahGeo 5 is out, adding a bunch of powerful features to the OS X photo geotagging app. New features include Places (saved coordinates and location names), drag-and-drop geocoding, Google Earth/Maps/KML export, direct camera import, and map searching.
Writing Workflow 2016, Part 2: Citations, Preview, and Export using Zotero, Marked 2, and Pandoc
A great rundown of a workflow for anyone who wants to incorporate citations and more advanced Pandoc features into Marked 2.
A free messaging app that combines chat and messaging services into one application. Slack, WhatsApp, WeChat, HipChat, Facebook Messenger, Telegram, Google Hangouts, GroupMe, Skype and more.
A Tweetdeck-style desktop client for GitHub. 15-day trial, $9.99 to purchase.
AppLandr - Beautiful landing pages for your mobile apps
Generate good-looking landing pages for mobile apps simply by providing the app’s store URL.

Udemy courses for 30% off

[Tweet : ADN : nvALT]

Udemy is offering a 30% discount on all of their courses for the duration of April. You can use the code TERPSTRA at checkout to apply the discount.

If you want to learn new skills and concepts on-demand, at your own pace and on an amazing array of devices, Udemy has both master and mini courses on everything from programming to photography. Take a look!

Just take a look at the available courses, and use the coupon code TERPSTRA at checkout.

Friday Freebie: Infographic Icon Set

[Tweet : ADN : nvALT]

For your Friday design pleasure, here’s a free set of infographic icons from 1001. The graphics include avatars, business charts and icons, mobile devices and more.

The download includes the following formats:

  • AI
  • EPS
  • PNGs (1024 x 1024 transparent png image of each icon)
  • SVG (individual SVG file for each icon)

Download the Infographic Icon Set.

The pack is licensed under a Creative Commons Attribution 3.0 Unported License.

Visit 1001freedownloads.com for more!

Web Excursions for April 13, 2016

[Tweet : ADN : nvALT]

Paid access to Instapaper’s web scraper. I’d like to incorporate this into Marky, but the cost wouldn’t even out. Still, one of the best tools available for finding the actual content in a web page.
Command - Chrome Extension
An open-source extension for Chrome that adds slash-commands (e.g. /giphy or /selfie) to text fields in your browser. If you’re a Slack or IRC user, you’re familiar with the concept already.
Printing TaskPaper 3 documents with Marked 2 CSS templates
A script for printing TaskPaper 3 documents via Marked 2 using some automation and CSS magic.
AnimateMate - Animation Plugin for Sketch
Create animations directly in Sketch.
WrapAPI: APIs for the whole web
Build an API on top of any existing website or find an API for a site that you need.

The TextExpander subscription snafu

[Tweet : ADN : nvALT]

Smile switched the TextExpander business model to a subscription plan last week, and the response was close to vitriolic from the community. I decided to hold off on saying anything until I had a bit more info.

Let me start by saying Smile is a current and long-time supporter of both my blog and my podcast, Systematic, and I’m a long time user and lover of Smile’s software. I will avoid bias as best I can, but a tilt in favor of Smile and its developers is going to be hard to avoid. Fortunately, there’s great news that I’ll get to in a moment.

The major mistake in the announcement of the new model was a failure to explain the benefits of it, or to provide any major new features along with it that would be relevant to individual users. The touted benefit of the subscription model was group sharing, which is really an enterprise feature, and it felt like individual users were being forced to pay for something they didn’t require.

TextExpander users have been quite satisfied with the current Dropbox/cloud sync for their snippets. What Smile left out of the marketing was that the current system had hit some limitations, and the move to a hosted service opened a new world of possibilities for feature development. Easy sharing and updating of snippets between users (without having to have a hosted URL) is only the first benefit; it also makes possible improved compatibility between platforms (Windows version in beta), things like Zapier and IFTTT integration and automation, and a host of new features they’re excited about (but can’t share yet).

The new model also ends the repetitive upgrade system. Once users are on the subscription plan, updates will come seamlessly, frequently, and without extra charge or major version bumps. Most of us have been upgrading regularly at a cost that comes out to about the same as a year-long subscription.

At face value, the switch was a jolt to me as well, so I understand the anger from current users. I think that a clearer explanation and a slower upgrade path would have made things much smoother. Apparently Smile, in retrospect, thinks so as well, as they’ve just announced an update to the release.

Previous customers gets a 50% lifetime discount on a subscription plan, and can opt to do a monthly plan at $2.08/month while deciding whether to invest in the yearly plan.

TextExpander 5 for Mac and TextExpander 3 for iOS (standalone, Dropbox/iCloud Drive versions) will be available (and supported) on a continuing basis.

With a small pricing change and the promise of continued availability and support for standalone versions, I think the door is open for a smoother switchover. Once development based on the new sync platform begins to offer more compelling features for individual users, I hope that the subscription model will become attractive—rather than upsetting—to those who already love TextExpander.