Welcome to the lab.

I’ve come up with a lot of ways to see what’s taking up space in my directories from Terminal over the years. I’ve finally built one that covers all the little niggles I’ve had in the past.

Let’s start with the other ways you can do this.

du

Since we’re talking about disk usage, the obvious choice is du, the “disk usage” command. To see the filesize of every file in the directory you can run du -sh *. The -h switch tells it to output human readable sizes, so it looks like:

4.0K	test.rb
4.0K	token.js
8.0K	utils
1.2M	webexcursions

This is pretty close to what I want, but it can’t be sorted by size. Also, du reports in blocks (512B per segment), so if you’re interested in accurate readings on files under 4kb, it won’t do it.

ls

You can also use ls -l to list all files along with their file sizes (and a whole bunch of other info). You can sort by size with -S (or -Sr for reverse order), and -h works here too to show human readable size formats. So that’s closer to what I want, but there’s a whole bunch of irrelevant info as well as the fact that ls isn’t going to report the total size of directories (all the files they contain added together) the way du will.

ncdu

I also have to mention ncdu, an ncurses utility that’s excellent for exploring disk usage. It’s overkill for what I want, but worth checking out (and available via Homebrew, brew install ncdu).

My Solution

You have no reason to recall this, but I’ve tried to solve this in the past. I wrote a bash function called sizeup that would do the trick. It’s super slow, though, and does things the hard way. So I decided to put ls and du together with some of my own sorting and formatting to get fast filesize info. I call it sizes.

Installation

I have the script posted in this gist. Save that file in your path, name it sizes, and make it executable (chmod a+x sizes).

Usage

To use it, just run sizes. You can optionally pass it a directory, e.g. sizes ~/Desktop, and it will operate there. And for whatever reason, I added help to it, so sizes -h will show you the obvious lack of other options.

The script will output a listing of all of the files in the current directory with sizes in bytes, kilobytes, megabytes, etc., calculated to 2 decimal places. It includes hidden files and reports actual sizes of directories (without traversing them). Output is colorized, with colors ascending from blue to red based on file size, and filenames colorized to indicate regular files, directories, and hidden files.

On any directory containing under 20GB it’s quite fast. Large directories can take a while to calculate, but you’d have the same delay using du directly.

How It Works

The script starts with an ls -l listing (actually ls -lSrAF) of the directory, using Ruby regex to extract the size (in bytes) and filename from the output.

Then it detects directories, which will be insufficiently reported by ls, and passes them to du to get the block-based filesize of the combined contents. It multiplies the size by 512 and gets as close to an accurate byte reading as possible. (It should be noted that the GNU coreutils version of du has a -b switch that will report in bytes. I wanted to make this without additional dependencies.)

The sizes are humanized, colorized, sorted, and output with the filenames and a total for the directory at the bottom.

I could easily be wrong, but I assume there are other people like me who want to find the space hogs without decoding too much output or loading up DaisyDisk. If so, enjoy the script.

I joined David Sparks and Rosemary Orchard on episode 20 of the Automators podcast. It was a riot, and made me realize exactly how nerdy I am about automation and its peripheral nerdery. One of the things that came up was my htaccess trick for handling Open Graph metadata on my blog. I got a bunch of questions about that, so I’m writing this up to explain.

If you’re unfamiliar with Open Graph, it’s a protocol which allows you to use meta tags in HTML to explain to various services what a page/post is, what image should represent it, and other information about a page. When you share a URL on Twitter or Facebook and it shows an image, summary, author information, etc., Open Graph is what allows the creator to control what’s shown.

Facebook and Twitter each have their own specs for preferred image dimensions and minimum size. I won’t go into all of those details right now, but if you’re setting up a system for yourself you’ll want to search the web for the latest information (it changes now and then). Different Open Graph tags target specific services, so I need different images for each service.

Here are my current sizes generated:

• _tw (Twitter): 715x383
• _fb (Facebook): 476x249
• _sq (Square, Twitter small size): 158x158

My setup automatically generates all of the necessary sizes from a template I use, naming each one with a suffix for the particular service it’s for. Sometimes, though, I don’t have a certain size available, which is where the .htaccess trick will come in.

In my templates I generate a standard boilerplate based on the name of the primary image for the post, appending the suffixes expected:

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="[url/path]/image-filename_tw.jpg">
<meta property="og:image" content="[url/path]/image-filename_fb.jpg">
<meta property="og:image:type" content="image/jpeg" />
<meta property="og:image:width" content="476">
<meta property="og:image:height" content="249">

Twitter doesn’t have specific tags for width and height, so those only apply to the Facebook image. My dimension tags are created using a call to sips when the static site is generated.

So what happens when a file with the appropriate suffix doesn’t exist? The .htaccess has a cascade of fallbacks, so when the image specified is requested but doesn’t exist, something gets served back. It checks to see if each requested filename exists, then rewrites the filename as the next option, repeating until finally it just falls back to serving the original image from my post.

For example, my current Web Excursions header doesn’t have a _tw version. I actually added that size since I created that image. So while the HTML specifies https://cdn3.brettterpstra.com/uploads/2017/03/web-exc-map_tw.jpg as the Twitter image, when you put that in a browser, you’ll actually be served the Facebook version: https://cdn3.brettterpstra.com/uploads/2017/03/web-exc-map_fb.jpg. Which is close enough in this case, each service will crop as needed. Having the right dimensions to begin with simply gives you control over how the image appears.

Here are the rules in my .htaccess file:

# Image handling for open graph meta

# Try _fb if _tw doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*)_tw\.(jpg|png|gif) $1_fb.$2 [L]

# Try _sq if _fb doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*)_fb\.(jpg|png|gif) $1_sq.$2 [L]

# Try _lg if _sq doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*)_sq\.(jpg|png|gif) $1_lg.$2 [L]

# Try @2x if _lg doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*)_lg\.(jpg|png|gif) $1@2x.$2 [L]

# Fall back to base image file if no @2x
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*)@2x\.(jpg|png|gif) $1.$2 [L]

By the way, this has the side effect of letting xxx@2x.jpg fall back to xxx.jpg if the @2x doesn’t exist, so I can always request @2x when serving to retina displays, even if I haven’t created one.

In summary, this system allows me to easily provide different images for different services, but gracefully handles cases where I don’t generate all of the needed sizes without me needing to modify meta or templates each time. The htaccess part of the trick is really just the tail end, but enough people asked specifically about it that it seems worth sharing. Hope that helps, feel free to contact me with any specific questions!

I work on a lot of different coding projects. Websites, front end and back, Mac and iOS coding, Ruby gems, scripting, design projects. While I’m working on a project, the build, deploy, and other development processes I set up become second nature. Once I’ve moved to another project or 20, I’ve learned it’s really easy to forget how I had it all set up. Maybe I was using CodeKit, or maybe I had gulp-watch set up, maybe everything is in a Rakefile…

As a result, I take notes whenever I set up Grunt or Gulp, add npm tasks, build out a rakefile, or just create some shell scripts to automate my processes. Just a reminder of what the process is and what tricks I’ve been up to. The notes are saved in the root directory, and I add the notes to the project’s git repo; they can help me save time explaining things to any collaborators, but mostly they’re just there so that when I dig the project up a year later, I don’t have to dig through all of the config files to remember what’s what.

To that end, I wrote a simple script to find these build notes and show me all or any given section of them. It relies on there being a file in the current directory with a name that starts with “build” and an extension of “.md”, “.txt”, or “.markdown”. I usually call mine “buildnotes.md,” but it will find anything matching those criteria.

The sections of the notes are delineated by Markdown headings, level 2 or higher, with the heading being the title of the section. I split all of mine apart with h2s. For example, a short one from the little website I was working on yesterday:

## Build

gulp js: compiles and minifies all js to dist/js/main.min.js

gulp css: compass compile to dist/css/

gulp watch

gulp (default): [css,js]

## Deploy

gulp sync: rsync /dist/ to scoffb.local

## Package management

yarn

## Components

- UIKit

The script isn’t terribly advanced. It expects there to only be one header level used to split sections. Anything before the first header is ignored.

Sometimes I write more detailed notes, but the above project was pretty straightforward. I did, however, end up with config files from multiple package managers around from my discovery phase, so until I clean it up you couldn’t just look at the directory and tell what package manager or build system to use. Ultimately, I just need enough info to know where to look for more details. If I know that I was compiling my css with gulp and compass, I know I’m looking for a gulpfile.js and a sass folder to start editing the CSS.

To use the tool, save the script to a file and make it executable (chmod a+x buildhelp.rb). I alias the script to build?, which makes it really easy to remember (I just ask the question, “build?”). To do so, just add alias build?="/path/to/buildhelp.rb" to your .bash_profile or wherever your aliases are stored.

Now with a build notes file in the directory I’m in, if I run build? in the root folder, it will output a colorized version of that entire file. I can also get specific sections by including the section name (case insensitive, only the first few characters needed to match).

Run build? -s and it will list all of the sections in the file:

- Build
- Deploy
- Package management
- Components

Then run build? dep to get just the Deploy notes.

I know it’s a bit silly, but keeping a consistent system as I move through different projects helps keep me sane and avoid wasting time tracking down my tools. If it sounds helpful, grab the script from this gist.