The problem with tinyurl and similar services (recycled)

Funnily enough, not long ago on a mailing list I’m part of I asked people to use TinyUrl, forgetting what I wrote a while back about the problems with this service, a subject that has resurfaced these days and that could turn into a serious security problem. Here it goes, recycled and with a few improvements, my post from December 27, 2007:

The problem with tinyurl and similar services

This post was sitting in my drafts, and I’ve now finished it in response to a question from FT via twitter,

If one day TinyURL or Dev.cl stop providing the service... do all the short URLs they created die?

The short answer is Yes.

Incidentally, when I tried to shorten the url of this post, dev.cl gave me back the url http://dev.cl/.

Now the fun answer:

Although short urls have several uses and advantages, they’re even good for compressing images ;) the truth is that there are several problems:

Jeff Atwood sums up several of these problems:

If the service disappears, the urls are broken forever. Just as FT fears.
The other problem is quality of service. A service of this kind that works only sporadically (with "outages") may actually be an even worse situation.
You lose any clue about the content of the url, since they are all homogeneous and boiled down to a cryptic code. This is bad for the user experience of whoever is clicking the link (and for security), because the url gives you no hint about where you’ll end up.
There are url redirection services that have been used in questionable ways, by spammers for example (that was the case with lnk.to).

Other problems that Atwood doesn’t identify:

The redirection introduces latency.
A shortened url can be used for phishing.
It becomes a layer on top of DNS.
It’s a security hole; it can be used to inject an XSS, for example.
It’s not a good idea to use this for things like ajax or web services.
If your firewall or reverse proxy blocks the service, that makes it unusable
The service may be full of junk, like urls that are no longer valid; having lost the clue, you don’t find out until you click.
If a url fails, you don’t know whether it was the shortening service that failed or the original url that has the problem.
Tinyurl doesn’t handle some "weird urls" well, the ones with characters like the comma (',').

Despite all this, short urls are useful in a number of contexts, as shown by their use on twitter, or for sending them in emails or SMS messages.

Why doesn’t Google provide this service?

That’s a good question and, as Atwood says, it would probably make the service more secure.
Maybe they’ve analyzed the problem and don’t see a business in it, or they don’t want to get tangled up with the spammer problem.

Although I suspect they don’t want to put themselves in such an uncomfortably hegemonic position.

One alternative would be to decentralize the service, a kind of P2P of short urls, an idea I haven’t explored but that could be interesting.

(What I have explored is the idea of eliminating DNS, but that will have to wait for another post.)

Perhaps an interesting analysis would be to look at what happens with these services and the principle that cool urls don’t change (laid out by Tim Berners Lee).

At first glance you’d think that by adding a level of indirection these services can help preserve that principle, but if we look more closely, they actually make the problem worse.
Indeed, if the original url changes, you also have to propagate that change to all the short urls on all the services of this kind.

Now, the problem of URL persistence is more complex still, and the collapse of tinyurl-type services is just one class of the problems associated with the persistence of URLs.

How would you implement a service like this?

Now, to wrap up, let me criticize Atwood’s proposal to use hashing as the mechanism for generating short URLs.

Although at first glance it looks good (an O(1) algorithm), it doesn’t seem like the most practical way to implement this service. The hashes he shows produce strings much longer than the ones we see in these services.

Better to go with an O(N log (N)) algorithm like the following:

There is a table URL, indexed by ID and URL_LARGA
A url comes in and you look it up by URL_LARGA, getting back ID or NULL.
If you get NULL, a new row is inserted into the table and the short URL will be the site’s root plus the ID expressed in base 64.

This lets us generate pretty short urls, and it’s easy to implement.
I don’t think it’s easy to find a hash function that generates shorter strings than this algorithm does.

As an extra tip, instead of doing the redirection with PHP or Python, I would go for writing an apache module in C, or in PERL. That would speed up the service by at least an order of magnitude (if not more). Or maybe by building my own web server. And finally, I would try to keep the table in memory, or at least a cache of the most visited URLs. :::

::: {.comments}

Did you like it? Sponsor us on Patreon Become a Patron! :::

::: {.comments}

Comments

::: {#disqus_thread aria-live=“polite”} Please enable JavaScript to view the comments powered by Disqus. ::: ::: ::: :::

[Published using Prosa - release 0.3.9]{.small} ::: ::: ::: :::