The lost art of telnet post


A long time ago, on an operating system far,
far away...




It was a period of protocol war.
Raw text connections sent
over the internet had
gained a strong following.

During this time, greater
needs and complexity arose
eventually leading to the
wide adoption of the hyper
text transport protocol

Knowledge of how the web worked
on these lower levels soon
vanished. It is up to you to
learn the old ways and restore
freedom to the internet...



There's something we once did back in the early days of the wild wild web, back before HTTP was the only game in town. It was called telneting, now known only in passing to some programmers, and a hand full of old school hackers that might still use it.

We lost something when we stopped using telnet, we stopped understanding how protocols are higher abstractions of raw text, how fragile things are, how simple they might be.

What is telnet?

Okay, so I'll admit telnet is actually the wrong technical term for it, but it's how a lot of early hackers learnt about protocols and did neat things. For all intents and purposes we're talking about sending raw text over TCP/IP. To be totally honest, this is a bit about internet protocols.

Fun and games

One of the coolest things I remember back even during the turn of the millennium was something called Multi-User-*, where the * was a place holder for "created Kingdoms" or "Dungeons". They were called MUDs, MOOs, MUSHes and MUCKs. They're a dying breed, born in the mid 1970s, but by now with these fancy web scripts and css tricks most people have entirely lost their interest in them.

There were no graphics or special data encodings, just a dumb box of text that you got to play with (for hours and hours). It may be hard to see why people would bother with these things. Eventually a handful of the servers did branch out and add images, but generally they weren't a feature.

I really got into them at an early age because one particular type of server had several programming languages exposed to the player. You could log in, and start writing code that manipulated the game world, and other users could walk into your part of the world and start to play with them. Anything you wanted you could program while playing the game with others, and export that code to another server somewhere. I actually ran one of the most popular sites on something called MPI back when I was about 12 years old, but that's long defunct now. Others played the game for the adventure of other user-created world and plots they wrote (that's right, we had user-created content as the focus of the internet back in the 1970s before I was even born, web2.0 and MMORPGs: go eat your heart out).


Well, wash my arm! Stren Withel will rue the day he eyed me over 
and started flipping tables at me!

It was a great environment for me to learn in, because it was a chat room (like IRC), it was a game world that I could explore and interact with.

The above is a snapshot of the Discworld MUD, where the theme is based on the famous Discworld series by Terry Pratchett! It's been on-line for almost 20 years now, and there are roughly 50-150 players logged in at any time.

The MUCK I was a big part of was actually the NMC project (a fork of a popular MUCK server). The only NMC server around right now as far as I know is Redwall MUCK, which you might enjoy as it has a web-based interface that lets you connect from a browser. One of the really cool things with that MUCK is that Brian Jacques, the series author, gave his permission for it to operate before his unfortunate passing earlier last year.

We don't really have this sort of thing on the Web as we know it. I mean, yeah sure, we have crummy phpBB installs where some people role play or do fantasy football. If you want a cheap laugh you can hop over to Kongregate and play Don't shit your pants, but you know what? It's not quite the same thing.


Sweet, I got a crown after unlocking 
all achievements.


There was something more engaging about using my imagination with 10-150 people, all of us sharing and creating. Somewhere along the line people started enjoying Skinner box rewards from killing sheep in MMORPGs for ten to fifteen bucks each month is more than a game that's text-based, and you have to create your own world and plot. Anyone who's experienced an addiction to NetHack knows just how fun these games can be.

Knowledge

One of the really cool things that existed was the gopher network. Before Wikipedia, we had a number of servers with networks serving to normal gopher clients. We still have a few today.

It sounds a bit silly, but I wish the idea of what gopher was ended up winning -- if it had, we wouldn't be in this "over-design-everything" mess we're in now. You can check out the gopher manifesto over here, which honestly reads more like a post mortem than anything. To summarize both of those, if we had gopher we would have websites that deliver pure content without any of the design chrome, or spam.

Pure media (sounds, videos, graphics, text, whatever) without needing to force a heavy markup layout on us to send the content. It's what the static web needs to be: just the knowledge, and not the sparkling prettiness of bad web design design and spam-filled "news" sites that floods the net today (one paragraph per page with ten ads and irrelevant noise). It's tragic the protocol died so long ago. Had it evolved with the same effort that HTTP had, we may have seen a very different net today, a cleaner one with less spam and marketing adds flying around. Perhaps one of the coolest things: it would have made this whole "mobile site" thing a total non-issue; we're only delivering the content, not the flood of menus, pretty backgrounds, background music, and flashing animated advertisements.

There's still some gopher servers out there, but the unfortunate part is that the technology has remained in near-stasis since 1991, so to get an appreciation for it you need to think you're back on a dial-up modem and JavaScript won't be invented for another four years.

Using lynx to browse a gopher of everyone' favourte web comic.

There are a few knowledge and communication based protocols floating around, like IRC, which has survived very well, and as most technically inclined people have likely used it before, so I won't be going into detail there.

Email is another great one. What could be simpler than SMTP? It has an interactive help system, and you even greet the service with "HELO" and lets you write emails on-the-spot, not a single data burst.

For fun, for profit.

Knowing all of this is interesting, sure, but what do you really get out of it?

Well, for starters, you gain an appreciation for how the internet is not the web, and especially not the web browser... it's so much more, so much richer, and has so much greater potential than the web! If you understand telnet you understand the Internet isn't a magic box that renders HTML strings into pretty pictures, that's only what happens after the internet takes place.

You understand the internet, you understand that every time you open a web address you send raw text that looks like "GET /Resource" at it's very core, there is no magic going on, it's not a closed black box that is beyond your understanding. It's so dead-simple you could make something just like it.

WireShark is a great tool for inspecting network traffic.
This is some of the raw text that was sent when I loaded Reddit.

So why did we suddenly stop writing cool and inventive protocols and stuff everything into an HTTP service? There's a few good reasons. All of these mixed protocols lead to total madness on the web, we didn't really have a common ground to do cool things on, you needed lots of different software to do lots of different protocols.

Of course, the web browser was originally meant to be the common user-interface for many protocols, it ended up being the HTTP/FTP browser. HTTP didn't end up being the best tool for everything but merely a tool for everything. Today we we have the SOAP monster and RESTful API services, along with some strange form of APIs that aren't either of these things but still run if the sun and the moon line up the right way. HTTP is nice in that it's brought a lot of normalization to the internet, but it's also halted innovation from doing something that isn't an HTTP/HTML driven web. Something that you may find entirely amazing is that over twenty years ago some smart people at XeroxPARC were talking about social media and personalized user interfaces: two things that are only recently present on the web. They were talking about doing this on the protocol level with something called WAIS, and if you have an hour you should watch this lecture from 1991 (right when the web started) and ask yourself why has it taken so long to finally get some of things?

We don't always need to write code for the HTTP/HTML internet to do some fun hacking projects, but it never really happens. My guess is that most simply don't realize they can make their own protocols, or cannot see a reason to to replace HTTP to try and accomplish something else in a different way, the same way FTP, Gopher, SMTP, or even IRC do. After all, all of those protocols can be emulated in functionality through HTTP with a RESTful or SOAP API.

Not enough people understand them

Not because they're complicated. As you've seen here, some are dead-simple. They aren't understood because they aren't thought about, because they're out of sight: it's hidden away, tucked inside the internals of your browser, email client, or any other software. It just works. HTTP/HTML was reaching a tipping-point back in 2005 where we had a hard time writing interactive apps like desktop software on the web, but the advantages of the web were promising. If AJAX and JavaScript hadn't come in and matured the user experience of the web as we knew it, we may have very well ended up with a new protocol already that would have done animations and rich interfaces as part of the protocol and encoding.

What am I suggesting? Learn the protocols. I'm willing to write about any specific protocol in more detail if there's a high demand for it. You can write a POP3 email client in less than 250 lines of code.

What would I really like to see? I wish for more hacker-inspired innovation in the protocol arena, where people just invent things only to push boundaries and see how something else might work. I want this for a few reasons. Politicians decided hyper-linking to pirate content is illegal, some decide blocking DNS makes sense, others say encryption is illegal... those of us who understand the internet truthfully understand how asinine these policies are. The other reason I wish to see more playing around is simply so we don't keep working ourselves into one corner of technology. The web is slowly becoming one protocol which was never designed with the entire web in mind. We're creating a feature creep simply because we push so hard to get everything into a browser that only supports HTTP and HTML.

What else might we have if we didn't have the HTML/HTTP web?
  • We already have the ability to build faster websites, if we could only have a protocol that sent pre-computed DOM trees, rather than HTML documents.
  • We could send multiple documents at once-- rather than sending a document that describes more documents to request to make back to the server that tells us what requests we need to make.
  • We could make a protocol specifically for discussion sites, so places like Reddit might exist in Telnet, unwrapped from the chrome of the web, and can work as a messaging service.
  • We could make everything a JSON object, because that's the hip thing right now.
  • We could send content in TeX format.
  • What else would you do?
We don't have new protocols because of the inertia HTTP has built up, but that doesn't mean you, me, and all the enthusiastic hackers around here can't take our internet back from the people who don't understand it.

Categories: innovation

Tags: protocls, telnet