I found a great new service via Smashing Magazine today: historio.us. It’s a bookmarking service with some seriously impressive twists.

I’ve been using Delicious with Delibar for quite a while now. A while ago, I stopped archiving every web page I thought I might need someday (yes, I used to do that) because I’ve realized that Google’s current speed and accuracy have made my local data store obsolete. That’s a longer post for another time. The problem that arises when relying purely on web search is knowing what information you need, knowing that you’ve seen it before, but not being able to piece together the search to lead to that specific post or page. Delicious tags help tremendously, but nothing beats a full-text search. Limit that search to selected items from your own history, and you’ve got an amazing memory enhancer.

Historio.us lets you bookmark and tag pages, and then stores a cached version of the page and a full-text index. It’s a lot like HistoryHound, which I wrote up for TUAW a while back, but it offloads the bandwidth, processing and storage requirements to the cloud.

There are two things I’d love to see improve:

• I wish the search was a little more savvy. It does well, especially with “tags:” searches combined with keywords, but fails to handle fuzzy matches very well. Substituting “other” for “another” can lead to entirely different results.
• I wish the API would let me pull recent bookmarks the way I do with Delicious for local tagging purposes — not a huge deal.

The Historio.us bookmarklet is simple and effective, and the current API does have everything you need for creating extensions and addons for bookmarking. The bookmarklet functions much like the Instapaper bookmarklet, and you can trigger it on any page to effortlessly add the page to your index.

You can also publish a personal search so that others can search your bookmarked pages, and for paid subscribers there’s a “read later” feature that lets you mark pages as unread, much like Pinboard.

Historio.us has a free version, but it’s lacking many of the more interesting features (like Read Later). You can get the full version for $2.99/month or an annual fee of $19.95. Seems worth it to me, so I’m going to give it an extended trial and see how things go.

Related posts:

1. I adore Pinboard
2. Auto-link text service updated
3. Automated search and link text Service

Originally posted on BrettTerpstra.com at Quick Link: historio.us

Why would you want to do this?

First, it adds Spotlight search abilities to your Delicious bookmarks, and, if you use OpenMeta tags, it integrates a tag-based search into any OpenMeta application. The second half of that is, in my case, I bookmark in different ways for different purposes, and it’s nice to have everything integrate into a full search in the end. For me, that search is primarily tag-based, thus the OpenMeta tagging. When I want to share a link, I use Delicious (via Delibar), and when I just want to save it for my own reference, I use OpenMeta (via Tags.app). This script is part of a system which blends the two, removing the need for redundant tagging and bringing my Spotlight searches together.

Read on for the new script, and installation instructions.

At the top of the script, you’ll find a section that looks like this:

-- CONFIG
-- delicious username
property _user : "username"
-- delicious password
property _pass : "password"
-- POSIX path to *existing* folder, manually create the folder if needed or point to one that does exist
property targetFolder : "/Users/OSXUSERNAME/Library/Caches/Metadata/Delicious"
-- use Tags.app to apply OpenMeta tags based on Delicious tags
property useTagsApp : true
-- add thumbnails using http://hasseg.org/setWeblocThumb/
property useWebLocThumb : true
-- END CONFIG

You’ll need to set these properties to your own information, but after that the script should take care of itself. Username and password are your login info for Delicious.com. Obviously, it won’t work without those. The targetFolder property should be set to a folder that already exists or that you just created for the purpose. The script will not create a folder that doesn’t exist. I recommend creating one in ~/Library/Caches/Metadata/Delicious, and just substituting your user name in the targetFolder config.

The last two configuration options determine whether posts are tagged with Tags.app and/or thumbnailed with setWeblocThumb. I recommend doing both, but if you have other purposes for the script, feel free to set these to false. In order to use either option, you’ll need to have the appropriate application installed. Tags is a $29US application that makes tagging everything from URL’s to iPhoto pictures, as well as files and folders, a breeze. Worth every penny, in my opinion. setWeblocThumb is a free utility that captures thumbnail images of the bookmarked page and applies it to the webloc file. It makes for very nice presentation of search results and bookmark folders.

The last thing you’ll probably want to alter is the first line after the end of the config. It sets the ‘date from’ parameter default to 1 hour prior, in Central Standard Time. If you’re not stuck in the middle of the country, you’ll want to edit the –v+4H part to a + or — number more indicative of your own GMT offset. It’s not a huge deal, this current script will grab everything after this date on the first run, but will record the date of the last bookmark it finds and use that for the next run, and so on. If you want to get all of your Delicious bookmarks on the first run, set this to something long, long ago.

I’ve smartened the script up a bit from the previous version, and solved some of the errors that caused the bookmark save to fail on certain filenames. It also checks for existing bookmarks before writing new ones to disk, avoiding duplicates.

As with the last script, you can use Lingon to set up an hourly run for the script in launchd, or even just set it as an alarm in iCal and use that to schedule it, but I’ve found a better solution. I use Delibar for bookmarking, and it writes out to an xml file in ~/Library/Application Support/Delibar. I used Lingon to set up a launchd process that watches for changes in that file and runs the script. Seems to work really well, at least so far. However you choose to run it, it will keep storing the latest date at the time of retrieval, so you can even just run it once a day (or week, or month) and it will find just the newest bookmarks.

Without further ado, here’s the script in its entirety, commented for readability:

Updated: Thanks to Matteo from Shiny Frog for tipping me off to the fact that the API rate limiting I was running into was due to the lack of a user agent. This has been added to the curl call in the script, testing will be much smoother now!

#!/usr/bin/osascript
-- CONFIG
-- delicious username
property _user : "username"
-- delicious password
property _pass : "password"
-- POSIX path to *existing* folder, manually create the folder if needed or point to one that does exist
property targetFolder : "/Users/OSXUSERNAME/Library/Caches/Metadata/Delicious"
-- use Tags.app to apply OpenMeta tags based on Delicious tags
property useTagsApp : true
-- add thumbnails using http://hasseg.org/setWeblocThumb/
property useWebLocThumb : true
-- END CONFIG

set _date to do shell script "date -v+4H '+%Y-%m-%dT%H:%M:%SZ'"
-- returns a date 1 hour ago (CST) in the format required by the twitter API.
-- you can adjust the date based on your time zone and the frequency that you run
-- the script by editing the -v+4H part of the date command (e.g. -v-1H for GMT - 1 hour)
-- Note that the Delicious dates run on GMT, so you have to account for your time offset
-- For me (CST), this means -v+4H if I want to go back one hour
try
    set _fromdt to do shell script "defaults read com.brettterpstra.getdelicious fromdt"
    -- if the last update time is stored in defaults, use it
on error
    set _fromdt to _date
    -- otherwise, use the date we just set
end try

set bookmarks to do shell script "curl -A \"OpenMetaTagger/1.1\" https://" & _user & ":" & _pass & "@api.del.icio.us/v1/posts/all?fromdt=" & _fromdt

-- gets the bookmarks starting with the date specified in _date
set _folder to POSIX file targetFolder as alias
-- turns the POSIX path in CONFIG into an AppleScript alias
set _output to ""

tell application "System Events"
    set xmlDocument to make new XML data with data bookmarks
    -- set xmlDocument to contents of XML file "Users:ttscoff:Desktop:deliciousbookmarks.xml"
    -- create the XML object
    set {astid, AppleScript's text item delimiters} to {AppleScript's text item delimiters, " "}
    -- store the current delimiter and set the new one to a space for breaking up the tags
    repeat with _post in XML elements of XML element 1 of xmlDocument
        set _url to value of XML attribute "href" of _post as string
        set _tags to text items of (value of XML attribute "tag" of _post as string)
        set end of _tags to "delicious"
        set _name to value of XML attribute "description" of _post as string
        set _title to do shell script "echo \"" & _name & "\"" & "|tr -cs /[^A-Za-z]/ ' '"
        -- strips some bad characters from the page title, solves some errors
        set _date to value of XML attribute "time" of _post as string
        try
            tell application "Finder"
                set AppleScript's text item delimiters to ""
                if not exists alias ((_folder & _title as string) & ".webloc") then
                    set webloc to make new internet location file at _folder to _url with properties {name:_title}
                    -- makes a .webloc file in our target folder
                    set _output to _output & "Grabbed " & _title
                    set AppleScript's text item delimiters to " "
                    if useTagsApp then
                        tell application "Tags" to apply tags _tags to files {POSIX path of (webloc as string)}
                        -- adds OpenMeta tags to our newly created file
                        set _output to _output & " and tagged with " & _tags as string
                    end if
                    if useWebLocThumb then
                        do shell script "/usr/local/bin/setWeblocThumb " & quoted form of (POSIX path of (webloc as string))
                        -- use setWeblocThumb to add a thumbnail to our webloc file
                        -- http://hasseg.org/setWeblocThumb/
                        set _output to _output & ", applied thumbnail"
                    end if
                    set _output to _output & "." & return
                end if
            end tell
        on error myError
            set _output to _output & "Error " & myError & return
        end try
        set {_url,_title,_tags,_name,_time} to {null,null,null,null,null}
    end repeat
    set AppleScript's text item delimiters to astid
    -- restore the delimiters
end tell
do shell script "defaults write com.brettterpstra.getdelicious fromdt " & _date
-- store the date of the last bookmark in system defaults
return _output

Open this script in your Script Editor

Related posts:

1. Delicious bookmarks and OpenMeta tags
2. Mirror your Pinboard bookmarks with OpenMeta tags
3. Save Safari tabs to Instapaper

Originally posted on BrettTerpstra.com at Delicious, Spotlight and OpenMeta tags, revisited

Unfortunately, one of my favorite apps right now, Delibar, doesn’t integrate with Spotlight or OpenMeta, despite the fact that it would be relatively easy to do. Delibar is an excellent (and sexy) menubar application for creating and searching Delicious (and Pinboard) bookmarks. A similar app, Pukka, pulls off the Spotlight part, but I still wanted OpenMeta tags that matched my Delicious tags for local searching.

I also use Tags.app for most of my tagging. OpenMeta tags allow you to group and classify files with simple tags, and provide a Spotlight-compatible way to search more intelligently. When you tag a website with Tags, it creates a webloc file in your metadata cache and applies the OpenMeta tags to it. I figured a similar approach would be feasible using AppleScript, and it seems to be working out great. Read on to see the script I’m using, and be sure to let me know if you improve on it!

What I’m doing is using curl with the Delicious API, and downloading everything that’s been bookmarked within the last hour. I’m running it with launchd every hour, so I don’t want to grab more than that every time it runs. It parses out the title, tags and url for the bookmark, saves it to a .webloc file, and tags it using Tags.app’s AppleScript commands. If you’re not running Tags, it could be modified pretty easily to work with the openmeta command line application.

I’m leaving the target folder up to the user, so when you define it in the CONFIG section, make sure it’s pointing to an existing folder. The only other config properties are your Delicious username and password (sent securely). As noted in the comments in the script, you can adjust the time/date it retrieves from in the first line after the CONFIG section.

The script is pretty well commented, so I won’t rewrite the whole thing here. Take a look, and then I’ll show you how I installed it.

-- CONFIG
property _user : "username" -- delicious username
property _pass : "password" -- delicious password
property targetFolder : "/Users/ttscoff/Library/Caches/Metadata/Delicious" -- POSIX path to existing folder
-- END CONFIG

set _date to do shell script "date -v-1d '+%Y-%m-%dT%H:%M:%SZ'"
-- returns a date 1 day ago in the format required by the twitter API.
-- you can adjust the date based on the frequency that you run the script
-- by editing the -v-1d part of the date command (-v-1H for 1 hour)
-- Note that the Delicious dates run on GMT, so you have to account for your time offset
-- For me, this means -v+4H if I want to go back one hour

set bookmarks to do shell script "curl https://" & _user & ":" & _pass & "@api.del.icio.us/v1/posts/all?fromdt=" & _date
-- gets the bookmarks starting with the date specified in _date

set _folder to POSIX file targetFolder as alias
-- turns the POSIX path in CONFIG into an AppleScript alias

tell application "System Events"
    set xmlDocument to make new XML data with data bookmarks
    -- create the XML object
    set {astid, AppleScript's text item delimiters} to {AppleScript's text item delimiters, " "}
    -- store the current delimiter and set the new one to a space for breaking up the tags
    repeat with _post in XML elements of XML element 1 of xmlDocument
        set _url to value of XML attribute "href" of _post as string
        set _tags to text items of (value of XML attribute "tag" of _post as string)
        set end of _tags to "delicious"
        set _title to value of XML attribute "description" of _post as string
        try -- it fails on some files, the try block lets us keep moving if we hit a problem
            tell application "Finder" to set webloc to make new internet location file to _url at _folder with properties {name:_title}
            -- makes a .webloc file in our target folder
            tell application "Tags" to apply tags _tags to files {POSIX path of (webloc as string)}
            -- adds OpenMeta tags to our newly created file
        end try
    end repeat
    set AppleScript's text item delimiters to astid
    -- restore the delimiters
end tell

Open this script in your Script Editor

Installing

The first thing I did was make a version of the script that excluded the date parameter (fromdt) in the curl call in order to download my entire history and process it so I was up-to-date. This took a while, but didn’t seem to cause any problems. The Delicious API is pretty quick to throttle you, so download the whole bunch in one API call or you’ll be in trouble.

After editing the config options, I saved the file as an Application Bundle (see addendum below) in my own apps directory (~/Applications). It doesn’t matter where you put it (or what you name it), as long as you know the path to get there. Then I set up a launchd plist to run it every 3600 seconds (hour). I highly recommend Lingon for editing all things launchd. It will make sense when you get there.

Addendum

In reference to the install procedure above, I’ve actually had more luck saving it as a text file to a scripts folder and running it with osascript through launchd. Here’s my launchd plist, which you can edit and load through the “Expert” section of Lingon’s editor.

Also, as I just added into the code comments above, I didn’t originally account for the fact that Delicious dates are in GMT, and my offset is –5 right now, so I have to use –v+4H to get 1 hour back, not –v-1H.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>org.brettterpstra.GetDelicious</string>
    <key>Nice</key>
    <integer>8</integer>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/bin/osascript</string>
        <string>/Users/ttscoff/scripts/getdelicious</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StartInterval</key>
    <integer>3600</integer>
</dict>
</plist>

Areas for improvement:

• Currently errors out on some bookmarks for unknown reasons
• Could have a first-run setting to automatically download and tag ALL bookmarks
• Could set the created date of the webloc file based on the date of the bookmark from Delicious
• Could provide more feedback or logging

Related posts:

1. Delicious, Spotlight and OpenMeta tags, revisited
2. Mirror your Pinboard bookmarks with OpenMeta tags
3. Natural language dates for TextExpander

Originally posted on BrettTerpstra.com at Delicious bookmarks and OpenMeta tags