On data ownership…

Something I’ve been working on in my programming projects recently is ways to allow users to use their data from outside my service, and to take the data they have in services I create elsewhere.
In the new era portability is king. If you don’t allow users to use their data how they want, your service is utterly useless and doomed to failure.

Whoa there, you say, them’s some big bold words! If it sounds like I’m including the large portion of current generation products people are familiar with as the target of my wrath, you’re right on the money. Let’s take a look at a recent real life example, MobileMe.

MobileMe offers some excellent syncing tools for those of us with Macs and iPhones. Over-the-air (push!) syncing of contacts and calendars is a great tool, especially for those of us with a desktop, a laptop, an iPhone, and an iPod Touch all trying to stay perfectly in sync. However that’s exactly where the scope of the tool ends. Want to share contacts/calendars with someone else as “joint ownership”? Screwed! Can’t do it. So MobileMe loses one user/evangelist to Google, where I can choose to allow another user to collaboratively edit my calendar. MobileMe ends up in the trash heap because the data I give it can only be used in the ways that MobileMe wants me to use it, and I have different ideas.
Google also frustrates me though because I would very much like to set up a group of shared contacts between Bev and my accounts so we could maintain contact synchronization, but that’s not supported through them either. I have a huge store of data, and I can’t even grant another user ACCESS to it. This is full of fail.

Obviously one of the core problem with this is common language. There has to be a standard protocol that is used for each type of data in order for sharing to really work. For inter-service data (like sharing contacts with another user of the same system) there really is no excuse however.

There is another form of data usage to consider beyond just sharing. A good example of the type of thing I’m thinking of is WebHooks, but I’m not completely convinced on their implementation.
A current example of the concept would be posting? to an online forum that lets you “follow” the thread. When you post you can check the box for “email me when someone replies”. Now whenever someone posts something you get a notification. WebHooks is like that, except instead of providing a simple email notification it allows you to provide a URL and a notification is posted to that URL. The notification contains whetever data the application designer wants it to. This may seem like a power-user feature, but once the concept is widely accepted it allows you to let websites (and the datasets they contain) to interact with each other in fabulous new ways.

This starts with us, the application designers. And this is why I’m so hot and bothered about the idea. If I don’t design my own applications to allow the sort of data interactions that I want from other websites I use. Once again though, the problem is standards. How do you output the data? Do you create your own refspec for the specific website/application? Where do you draw the lines?

Once again, the problem is standards. There’s no reason why any user should have both MobileMe contacts and google contacts. Ideally either service should allow the user to use, not just “import”, contacts from the other.

We’re not there yet, and I understand how people scoff at this idea, but I’ll say it again. Data portability and access is king. If you don’t let your users get to the data they’ve entrusted to you and use it in the ways that they want to use it, they’re going to abandon you.

On the failure of tweets, RE: “TXTing”

One of the core failures of Twitter is that it’s based around a flawed idea. Text Messages. 140 characters. Shorten everything and use slang. Sure it’s become cute to try and manage your communications in 140 character bursts, but WHY? Holy flaming fig trees, it’s just not needed! I mean,? you may as well base your internet service around telegrams sent via morse code. The text message is an outdated idea, well past it’s prime. It may serve some historical purpose, much like Morse Code still does, but it is irrelevant to the future. Would twitter be as fun if you had to use all upper case, couldn’t use punctuation or non-alphanumeric characters, and had to say “STOP” at the end of each thought?
“TODAY WORK SUCKED STOP SOMEONE ATE MY LUNCH STOP HASHTAG FML STOP” is lacking in so many ways… Locking your service to the rules of an outdated and useless medium just doesn’t seem smart!

What you say? “It’s used my billions of people across the world! How can it be outdated and useless?”
They use it because they can. Simple as that. The cell phone companies provide it, including it’s outdated limitations, because it’s what you’re used to and what they can charge for. Never mind that it would be trivial to replace every SMS client in new phones with a Jabber (XMPP) app and install Jabber->SMS gateways to support legacy systems. (Okay, maybe not “trivial”, but you certainly see my point..) XMPP is a FAR better protocol by any standard. The vast majority of phones released on the market today would easily support this sort of change, “smart” phones especially. (You could even transfer seamlessly from your mobile device, to your computer, and back to your mobile device, without ever missing a message!!!)

Quick history lesson, if you need it. SMS, the “Short Message Service” operates on the control channel of your cell phone. This is a special data channel your phone uses to keep in contact with the towers and is always operational. This allows your SMS messages to get through even in the most dire of coverage areas, and even when your phone is in use, but is also what “limits” it to the 140 (actually 160) characters. Yes, I will concede that these are useful things about SMS that do make it stand out somewhat. I still thing we can design our way around it’s limitations though. However, there is no incentive to do so because even new “web 2.0” companies (like Twitter) are observing the limits of SMS and working within them! There’s nobody out there saying “hey, could there maybe be a better way?”

But my hatred of corporate greed and tyranny isn’t my only reason here! My love of language and clear communication weighs in rather heavily!

I am extraordinarily tired of slang and shortened “can I buy a vowel please Pat” words being used where they’re utterly unnecessary. And SMS is almost completely to blame for this change.
Now I know I sound like your college English teacher, railing against the evils of today’s youth… However, I think we can all get behind the idea that anyone who shortens “email” to “eml” in a non-space-constrained forum (in this case, facebook post) is a raving git. It’s completely and utterly unnecessary and the gross misuse of these sorts of shortenings obfuscates countless messages every day! (And by perpetuating mythical 140 character limit, Twitter perpetuates the cycle of social decay..)

Perhaps I just can’t help myself. I like to understand what people are talking about, and I very much like it when they understand what I am talking about. English has enough screwed up about it as-is, no need to intentionally make it worse.

Also, stick it to the man. That cell phone company is overcharging you out the ASS for those 140 characters, for no reason other than they really really like taking your money. Don’t fight the future. Evolve or die.

Why no observer patterns? (Pt. 2)

Immediately after posting the last article I started thinking about how this could work. Let’s assume a RESTful Rails based project.

Things we want to observe: Resources.
Done! That was easy!

So: http://daedalusdreams.com/observers/ would show you a list of things you could “subscribe” to, and indicate which of them you were already subscribed to. This would require you to be logged-in (openID) and have your own site’s URL tied to your login. (Thought: OpenID metadata?) Then you could simply say “I am interested in blogposts, microblog posts, and any new images you post.”

Then I, as daedalusdreams.com, would have an option when I went to submit a blogpost: “Notify Observers?”. (Maybe it would only appear if this resource had people actually observing it?)

I should speak briefly on the core principle of this system and the reason I started thinking about it. People often think I am against “social networking”. In reality, what I am really against is personality fragmentation. I refuse to have a twitter AND a myspace AND a facebook AND a linkedin AND a flickr AND a del.icio.us AND twitpic AND youtube AND…. I AM daedalusdreams.com, and everything about me should be in this single place. (Subject to needing additional hosting/bandwidth, but that can still all be linked to daedalusdreams.com and found here.)

I’ve just discovered something called WebHooks.? I think I’ll go read about that for a while. Looks interesting.

Why no observer patterns?

The observer pattern is really quite handy:
“Hey, you, the dude responsible for managing Josh’s twitter posts! Let me know when he posts something new! My address is…”

That’s basically it. It works in Cocoa, it works in SproutCore (Javascript), it works in Rails.. Why has it never been taken to a wider scale?? I understand that pushing to users is difficult due to the “statelessness” of HTTP, but “push”ing between servers with static IP/Hostnames should be decently trivial.

REST, JSON, SSL, toss out any number of acronyms you want. It’s doable in a number of different ways. So why isn’t it common? Why can’t I subscribe to your blog and have it notify me when you make a new post? Forcing me to poll your RSS feed periodically to see if you’ve made a new post is, shall we say, fucking backwards and wrong!!

So what are the real challenges here?

  1. Callback address. What if a registered observer changes their callback URL? How do we address them?
  2. How do we authorize changes? Must research OpenID/OAuth and see if there are systems that might work for this.
  3. Sufficiently extensible and user-definable. No locking the user profile into ONLY having fields like “favorite pet” and “smoker y/n”. No limiting the language used.

What might this look like?? If I (joshproehl.com) want to obsevre events at daedalusdreams.com I could send this:


http://daedalusdreams.com/observer/create
POST
{
model:blogpost
callback:http://joshproehl.com/listener/
}

And then when daedalusdreams.com makes a change to the blogposts it would know to send this:


http://joshproehl.com/listener/
POST
{
source:http://daedalusdreams.com/
model:blogpost;
sourceURL:http://daedadalusdreams.com/blog/5
}

I’m having trouble visualizing both exactly how this would work, and why it might not. I think it may be time to start diagramming some things out.

Continue reading in Part 2