Update: GMail IMAP import screwy, at least for now

GMail seems to be re-timestamping email messages copied via IMAP, so IMAP is not an ideal way for me to import old Outlook messages, as I had thought. All the old messages appear new in list views and sort new. D'oh.



I added an update to the original item.



Update: Based on a comment from Stephen Foskett and some workarounds/fixes I read about in the GMail Google Group, I am hopeful I can get this working in the next few days. Jamming at work right now, though.

We're halfway to RSS replies (in reply to inReplyTo)

This past weekend I posted to this blog the idea of an inReplyTo RSS element to enable all kinds of cool features in feed readers and to civilize discourse on the Web.



Now I realize we're halfway there.



For many, if not most, RSS items, you can figure out what other items are being replied to simply by looking at the links within the body of the item ("description" in RSS or "content" in Atom). Whatever URLs are linked to are what the item is "reply"ing to.

Someone made Flickr for your MP3s!

So a few weeks ago I publicly yearned for a personal online MP3 locker -- basically Flickr for your music files. I even reserved tunesr.com.



Well, last night Arvin Dang posted in the comments a pointer to LaLa.com that looks almost exactly like what I wanted -- basically the look and feel of iTunes on the Web.





There seems to be one big thing missing, however: a bulk upload tool. When I tried to upload from my new lala.com account, it gave me a file prompt that would only let me select individual files. On the other hand, the stated minimum requirements in the Help section is Mac OS X 10.4, and right now at work I only have 10.3.9. I'll try again at home and see if things are any better under 10.4 (Though most of my files are on Win2k and that's not officially supported ... d'oh.)

GMail PST import solution -- sort of.

D'oh! My get-rich-quick scheme has been foiled!

On the (very) bright side, Stephen Foskett, a fellow pack rat, has figured out a way to upload your email archives into GMail that is going to save everyone a lot of time. No installing POP servers, converting to CSV files, or losing your email timestamps-.

You basically just turn on IMAP in Outlook and in GMail, and then copy emails into GMail by dragging and dropping onto the appropriate GMail folder. Uh, pretty simple and elegant. Way to make us look dumb, Stephen!!

Why I never thought of this, or Louis never thought of it, I have no idea. I guess we were fixated on PST files and forgot about IMAP, since it is so new within GMail.

Update: Actually, when you copy email this way, GMail stupidly timestamps it with the time you copied it, instead of with the actual times in the email header, just as with SMTP delivery methods. So copying emails from 2002, they show up as new, sort as new, summary timestamp in list views is new. Only once you click through to the message can you see the actual date and time.

Hopefully GMail will fix this. Until then, POP3 seems to be the only way to get something into your GMail account and have it sort and summarize the timestamp property.

Update2: Based on the second comment from Stephen Foskett in this thread and some workarounds/fixes I read about in the GMail Google Group, I am hopeful I can get this working in the next few days. Jamming at work right now, though.

GMail import from PST, mbox - my scheme

Get-rich-scheme: Make a website where people can upload their old Outlook .PST and Entourage/Netscape/Etc. mbox files. Then the site transfers the emails to their GMail account via POP3. Charge per GB or something.

(Second in a series.)

RSS killer feature - Replies, via in-reply-to element

RSS feeds would be so much cooler if items could declare themselves to be "in reply to" other items or URLs.

This would allow the following awesome features:

• Discussion threads within your RSS reader



• A "find all replies to this post" feature in your Web browser or RSS reader



• A "hide all replies descending from this post because I am SO TIRED of this topic" feature in your search engine or RSS reader



• A reduction in anonymous comment flames, since this improvement to RSS feeds would encourage people to post replies to their own blogs under stable identities they invest in

The implementation would be dead-simple: add to RSS the "In-Reply-To" header we have had in email since the birth of the standard. This header is the bedrock of "the best known algorithm for threading messages."

We just need an RSS extension and namespace to hold the element, which would probably be called "inReplyTo" or something like that, and of course would exist at the item level. If the extension namespace were, say, "threading," a couple of related RSS items in the same feed might look like this:

<item>

<title>Drupal development full steam ahead on linode</title>

<link>http://www.linode.com/forums/viewtopic.php?p=13246#13246</link>

<guid isPermaLink="true">http://www.linode.com/forums/viewtopic.php?p=13246#13246</guid>

<description>Well, I've been able to get off to a fast start here at linode. A very clean, lean and mean setup ...</description>

</item>

<item>

<title>RE: Drupal development full steam ahead on linode</title>

<link>http://www.linode.com/forums/viewtopic.php?p=13247#13247</link>

<guid isPermaLink="true">http://www.linode.com/forums/viewtopic.php?p=13247#13247</guid>

<threading:inReplyTo>http://www.linode.com/forums/viewtopic.php?p=13247#13247</threading:inReplyTo>

<description>I see that like myself, you use the GNU Midnight Commander - it sure makes setting up a server easier ...</description>

</item>

But related items wouldn't have to been in the same feed -- they could be on entirely different websites.

This whole scheme burst into my head as I was catching up on some RSS feeds this weekend and saw the following in Google Reader (click on image for full size):

This looked broken.

See all those "RE:"s?? They are replies in the discussion forum for the virtual server I rent. I need to monitor this RSS feed for downtime, changes and upgrades to the server, but I end up drowning in all the replies, because there is no threading to hide them away.

So much else about Google Reader is broken these days. But this one actually isn't their fault. Email has had threading built in since RFC 822 25 years ago as I noted above, but RSS has nothing like it.

I looked in the spec, and the closest thing is the "source" element. But this element refers to an entire feed, not a specific item or URL. Its purpose is to "propagate credit" for the information in the item. It looks like maybe it could be repurposed for replies. Is this against the spirit of the element? I am not sure.

My guess is that inReplyTo should be its own element, for clarity. If you have people using "source" to mean "in reply to," then how can you tell when people are using "source" to mean what it originally meant, "I got this news via Feed X?" You'd have to look at what the element points to -- is it a feed or an item/web page? That takes a lot of work on the part of the reader author. Really, you'd have to do a lot of fetching.

Also, I can envision a scenario where you'd want to use both "source" and "inReplyTo" -- I find out about "Post Z" from Daring Fireball, so I put the Daringfireball feed URL as "source," but my post is a big rebuttal to "Post Z" so I put the URL (or guid) for "Post Z" as "inReplyTo."

Anyway, I also skimmed the RSS extensions to see if this showed up anywhere and never saw anything like "inReplyTo." I thought maybe the Trackback extension would have it, but that's all about pinging. Nothing really in Dublin Core ("relation" I couldn't quite figure out). The Comments extension is just that -- all about comments to a post, not posts relating to one another.

I'm really not sure how to get this going. Obviously, I can't force all the blog software hackers to add this to their tools, and then force all the authors to add this info to their posts, and then force all the reader hackers to add threading to their readers.

I do have a sort of RSS generation tool written in perl that I could use as a filter to add inReplyTo to my discussion group feeds. But I don't want to go back to hacking on my custom feed reader, I like the goodness of Google Groups .... Maybe the RSS filter could mash each thread into its own ginormous item, so it would work in Google Reader.

Some day when I have more time, I suppose.

---

Update: This was discussed for the Atom feed syndication format, but it doesn't look like it showed up in the (draft?) proposal. One alternate possibility would be to do <link rel="http://inreplytoforfeeds.org/inReplyTo">http://foo.bar/meh</link> in Atom feeds. But I'm not sure having multiple link elements would be allowed under the RSS2 spec, and it's certainly not common in the wild.

Also, someone drew up a threading module for RSS1, but RSS1 is not in common use in the wild anymore, most people are building on RSS2 or Atom.

Apple decides I can't buy music

I went to buy the new Manu Chao album on iTunes. It turns out I can't buy it.

My computer can buy songs on iTunes, as long as they are locked to my computer using a complicated "DRM" system.

And until recently, my computer could also buy simpler song files through iTunes, the kind without DRM.

But now Apple has cut the price on those simpler song files, and apparently the only software powerful enough to deal with the INCREDIBLE complexity of a price cut on plain digital files needs Windows XP or Windows Vista, Windows 2000 is no longer good enough, even though it could handle the older complicated DRM files.

So if Apple wants to cut prices, it has to release a new version of its software, a new version so much more complicated it is incompatible with an entire operating system that ran the old version.

I've bought 115 songs through iTunes on this computer, and now that the files are getting simpler and cheaper, suddenly my computer's not good enough.

There's something deeply wrong with that.

Hey Apple: Thanks for making my brand new iPod so much less valuable!

7 suggestions to improve Google Docs

I've been using Google Docs to expand and revise my application for a News Challenge grant from Knight Foundation. Here are some obvious, needed improvements that leap out to me in approximately the first five minutes:

• Add stylesheets. There are certain combinations of text formatting I'd like to be able to apply repeatedly and consistently. Also, I'd like to be able to revise these styles. (Clearly, CSS should be used to build this feature on the back end. The hard part is the interface.)



• Open documents in the main window, not in new windows. It is really annoying that clicking on a document spawns a new browser tab or window. Docs should open in the main Google Docs window, replacing the Google Docs home page. It is easy to return from the Doc to the Google Docs home page should you decide to do so.



• Open a Google Groups group for Google Docs. If people have questions, suggestions or want to report problems, there is no clear central place to do so. If you click around enough in the Help pages you may be able to submit a question, but no one outside of Google can see it, and traditionally one does not expect a quick answer, or any answer at all, when making such an inquiry to Google. Blogger has a Google Group which provides a nice support function, allowing users to help one another, determine when a particular problem is widespread, and vent frustration.



• Backups. Allow them. There is no way to back up all your Google Docs. Other Google services lack comprehensive backup options, but most provide workarounds: GMail can be mostly backed up using POP access and Contacts export; Google Reader subscriptions can be backed up using OPML export; Blogger blogs can be backed up using the FTP publishing function or using Web spider software. There is no such workaround for Google Docs.



• Combined GMail-Docs search. It would be really cool if I could search my GMail and Google Docs simultaneously. If I could do this, I could keep my interview notes in Google Docs, which has revision tracking, instead of in the Drafts folder of GMail. I don't do this now because I don't want to have to run multiple searches to check email and voice interviews. If this combined search also searched my Google Reader items, all the better. Ditto with my Blogger posts! If I could customize what it searched, even cooler still. PS, I want to preserve the ability to only search email or only search docs, etc.



• RSS feeds to monitor shared docs. When someone modifies a Doc I share with them, or shares a new doc with me, it would be cool to get this info via RSS feed. Even cooler to have the option of turning off email notifications (which I'm pretty sure I only get for newly shared docs, not modifications to existing docs). Even cooler to be able to specify the granularity of what goes in the RSS feed (or to pick among feeds of varying granularity). Cooler still to be able to ovveride these settings (or add feeds) for specific docs.



• Unlame your blog. Right now the Google Docs "team blog", linked from the help pages, contains one entry dating to September, which consists entirely of a link back onto itself. [Update, Nov. 25 2007: The old lame blog, google-d-s.blogspot.com, now redirects to googledocs.blogspot.com, which looks like it has been regularly updated for some time now. Not sure why they didn't link to this blog in any of their help docs! They must have linked to it from some general Google Blogs directory or something, otherwise I don't know how anyone would have known about it.]
  

(With apologies to Louis.)

Google Reader is a slow, limpy, sedated way to read news

So I subscribe to the feed containing this item, and 46 minutes after it was posted it has still not shown up in my Google Reader account. I only found out about it through a Google News alert RSS feed.

Last week I noticed big delays with other sites in Google Reader.

Perhaps I'm extra sensitive to this because the reader I used to use, which I wrote myself in Perl and which ran on my own server, checked every single feed every single time you brought it up, using conditional HTTP GET requests. I was *never* getting anything other than the latest news from all my feeds (though I did consider inserting code to limit checks to, say, no more than one every 10 minutes).

If Google doesn't have the server resources to fetch feeds on demand so they are never more than 15 minutes out of date, it should create a browser plugin allowing us users to fetch the feeds ourselves on the client side.

Where is Flickr for your MP3s?

Whatever happened to online music lockers?

The free version of GMail now offers nearly 5GB of data storage, as much as my $150 iPod.

Meanwhile, Apple is selling more and more unrestricted music in the AAC format. These songs are not tied to a particular computer, unlike the songs that used to be sold through Apple's music store. And Apple recently cut prices, making unrestricted songs as cheap as restricted songs.

So why hasn't someone built a website where I can upload all of my music files and listen to them from any computer? At work, on my wife's laptop, in a hotel, etc.

The infrastructure is cheap. In fact, it's so cheap, I'd gladly foot the storage bill myself. Amazon now charges just 15 cents per month to keep 1 GB of data on its Web servers. Add in the cost to upload that data, and you're talking about $15 for the first year to dump the contents of my iPod onto the Web (falling to half that for subsequent years, since the music is already uploaded).

A smart company could probably offer the service for free and make money interspersing audio ads with your music. Or make online storage of unrestricted music files a premium upgrade to an existing product.

Bear in mind that I'm talking about music storage, not music sharing. Aggressive interpretation of copyright law is what doomed the original MP3 locker, my.mp3.com, seven years ago.

I reserved tunesr.com for just such a service, but amid my other projects have not made much progress on it.

Google Reader delivers news like a snail

Right now Google's RSS reader is lagging

• Fake Steve Jobs - 4 posts
  



• Daring Fireball - 2 posts



• Eater SF - 2 posts

And those are just the ones I'm aware of (and with 146 feeds, I can only check a tiny percentage, so the problem is probably worse than this).

Google has a great news reader, as long as you don't expect your news to be, well, new.