This is not a partial demotion, a ranking issue, or a temporary crawl problem. The entire domain is completely excluded.

In addition to excluding yorniq.xyz from search results, when we discovered the block, Bing was also placing what appeared to be a phishing attack against Neocities on the first page of search results. This is not only bad for search results, it’s very possible that it is actively dangerous. After complaints (it required several) they deranked the suspected phishing site, but yorniq.xyz results remain blocked, and it is possibly only a matter of time before another concerning site appears on Bing searches for Neocities (it’s easy to get higher pagerank than a blocked site).

In addition to the safety concerns, this also unfairly affects over 1.5 million independent websites hosted on Neocities, the vast majority of which are personal, artistic, educational, or experimental projects with no commercial or malicious intent. These are brilliant and wonderful sites with billions of human visitors per month and they don’t deserve to be blocked from an entire search engine for no reason.

We have repeatedly attempted to resolve this through Bing’s official webmaster and support channels, and a few internal channels. Despite these efforts, Bing has declined to reverse the block or provide a clear, actionable explanation for it. At this point, we have exhausted all reasonable avenues for remediation except public disclosure.

Because of this, we are recommending that Neocities users, and the broader internet in general, not use Bing or search engines that source their results from Bing until this issue is resolved.

In addition to Bing, there are other search engines that currently rely on Bing’s search results, including but not limited to DuckDuckGo.

If you use Bing or Bing-powered search engines, Neocities sites will not appear in your search results, regardless of content quality, originality, or compliance with webmaster guidelines. If any Neocities-like sites appear on these results, they may be active phishing attacks against Neocities and should be treated with caution.

What This Is Not

To be clear:

• This is not the result of widespread malware or phishing on Neocities
• This is not due to a technical misconfiguration that Bing has identified
• This is not the result of policy violations communicated to us in a meaningful way
• This is not because we haven’t made multiple good-faith attempts to contact Bing for resolution
• This is not because of low quality AI slop, of which Neocities has almost none
• This is not because of suborigin issues (Bing has documented that they understand suborigins)

Neocities allows users to publish HTML, just like every other general-purpose web host. We have always responded to valid abuse reports and comply with applicable law. In addition, we have an active and sophisticated moderation process that removes the vast majority of malicious sites pre-emptively before they are even indexed by search engines.

Why We’re Posting This Publicly

We did not want to write this post. We try very hard to have a good relationship with search engine providers. We would much rather quietly resolve this issue with Bing staff and move on.

But after months of attempting to engage constructively through multiple channels, it became clear that silence only harms our users. Especially those who don’t realize their sites are invisible on some search engines.

We decided to post this so that:

• users understand that their sites aren’t appearing on Bing or Bing-powered search engines
• web users can make informed choices about what search engines they can trust to be safe and reliable
• the situation is documented transparently and publicly

Our Recommendation

If you care about discovering independent websites made by real human beings, personal pages, and creative projects on the open web, we recommend using search engines other than Bing.

At the time of writing, Neocities sites continue to be indexed normally by most major search engines, including Google.

We will update this post if Bing reverses its decision or meaningfully engages with us to resolve the issue (if you work at Bing or Microsoft and can fix this, please contact us, we would really love to talk to you). Until then, we cannot recommend Bing or Bing-powered search engines to our users, where you will not even find this blog post in the search results.

So for example if you have a /my-favorite-songs.html, you can now load it with your browser by just typing /my-favorite-songs. Previously you could do this by creating a directory called “my-favorite-songs” and then adding an index.html to it (my-favorite-songs/index.html), but this will make that step optional. Every link to a web page with the .html on the end will still (and always) work, and will automatically redirect to the cleaner URL, so there shouldn’t be any issues with search engines or links breaking.

The main drawback to this is that it may affect local site editing. But if you edit your site locally and the cleaner links don’t work with your local server, you can still use .html in your links and they will redirect automatically on Neocities. You should also still use .html for the actual files when you create them on Neocities.

For the edge case where there is both a directory index and a root page, such as /my-favorite-songs.html and /my-favorite-songs/index.html, the slash will be used to denote the directory, so /my-favorite-songs and /my-favorite-songs/ will load each respective page.

This change also makes it slighty harder to move to other static web site hosting, but it’s easy to configure other servers to support this. If you run your own server and use nginx for this, this is the code you can use to make it work the same way (in the server {} section of your config):

if ($request_uri ~ ^(.*/)index$) {
  return 301 $1$is_args$args;
}

if ($request_uri ~ ^/(.*/)index\.html?) {
  return 301 /$1$is_args$args;
}

if ($request_uri ~ ^/(.*)\.html?) {
  return 302 /$1$is_args$args;
}

Enjoy the cleaner, easier to type in web page links! Happy New Year!

Support for file moving / renaming

For a long time if you wanted to change the path or name of a file on Neocities, you had to re-upload it. This was inconvenient for a lot of people, so it has been our most requested feature.

The reason this was lacking was because most upstream file storage providers don’t support renaming, only copying files. For sites with a lot of large files, this would have been difficult to implement and taken a lot of time for simple operations to complete.

We have put together a solution for file storage that prevents this problem from happening, allowing for file renaming and moving without having to do a full copy. So there is now support for file/directory renaming and moving, which you can do directly from the dashboard. The interface doesn’t yet support drag and drop, but we’ll be looking at implementing that soon.

Faster loading web sites: TLS 1.3, Brotli

We recently rolled out some upgrades to how we serve Neocities web sites that will improve both performance and security. TLS 1.3 reduces the “handshake” time of establishing connections, improving the speed of initial connection by up to 100ms in some edge cases. We also turned on Brotli compression support, which reduces the size of the data that needs to be transferred to web browsers by using better compression algorithms. This helps web sites load faster, particularly on lower bandwidth connections such as mobile phones.

Enjoy!

Continuing down this adoption path, I’m happy to announce that Neocities now has support for IPFS DNS records! That means that most of the 140,000+ sites that have a yorniq.xyz subdomain can now be accessed through IPFS via DNS.

You can see it in action at https://ipfs.io/ipns/testipfs.yorniq.xyz, or if you have IPFS installed, you can view it locally at http://127.0.0.1:8080/ipns/testipfs.yorniq.xyz.

This works by using TXT records with our DNS servers. You can run
dig TXT testipfs.yorniq.xyz in a command prompt to see it in action:

dnslink=/ipfs/QmdJtiWqGEDsUJqW41aTQcds7U2PzmLDWs8qQ346mNoPkF

Right now we’re only supporting IPFS immutable hashes (not IPNS/keypair ones), which means there is a short delay before DNS records update with site changes through IPFS, but we’ll reduce this time lag soon. In the interim, this gives our IPFS implementation its first taste of mutability.

The distributed web is still in the foundation stages, and things are constantly changing. We’ll keep testing new things and see how we can help improve the distributed web. Enjoy!

Some of it’s many features include:

• Easy to use - simple design, informative help screens for each subcommand.
• Push support - uploads an entire directory. Great for static site generators.
• HTTP persistence - the CLI reuses a single HTTP connection, improving performance.
• Check before upload - checks if the file data matches before uploading.
• Gitignore parsing - automatically skips files in root gitignore file for git repositories.

Try it out

If you’re running OSX or already have Ruby installed, run this command:

gem install neocities

If you get a “permission denied” error, you may have to run the command as root:

sudo gem install neocities

After installation, you can run it by executing the neocities command, which will display a help screen with a list of subcommands.

If you don’t have ruby (or aren’t using OSX), click here to see your options for installing Ruby for your platform.

As with most of our source code, the Neocities CLI is open source and available for review (and pull requests) on our github repository. Have fun!

Since the FCC apparently doesn’t care anymore, I did some digging and found a bunch of IP address blocks used by net neutrality opponents, and made something special just for them.

Starting today, if you’re at the US Senate, the FCC, the White House, or a Trump hotel, you will now see this instead of the normal page when you sign up for a Neocities account:

For people at US Senate, the FCC, the White House, or a Trump hotel, this is your new plan and your only option. It’s an expensive, ridiculously low-resourced option that costs $100/mo, simply because you’re using the internet from networks that are working to destroy Net Neutrality.

If you think this example of rate discrimination is scary, the reality of what’s about to happen is even scarier. Here I’m being transparent with it, allowing you to see it happen. Comcast won’t be - they’ll do it secretly in their datacenters. They’ll figure out who their competitors are for their crappy Video on Demand (VoD) service (hint: everything you know and love) and de-prioritize their traffic so it can’t reliably reach you. Or they’ll decide competitors like Netflix need to pay 10x more for peering costs than everybody else pays. Comcast’s VoD service and their partners, of course, won’t pay any of it.

Even with the weak Net Neutrality rules in place, Monopoly ISPs like Comcast already charge everybody on the internet 2-3x more for peering than it actually costs (which you ultimately pay for). As a consumer you get double taxed, from the ISP itself on one end, and from your favorite web site’s expensive bandwidth bills on the other. Gutting Net Neutrality will just give them a blank check to further gouge their customers.

We learned what happens when the railroad barons of the 19th century were allowed to do rate discrimination with monopolized utilities. Don’t let the internet go down this same dark path. Now is the time to start fighting back.

PS: If anyone outside of the IP blacklist wants to sign up for the Trump Plan for some crazy reason, it’s not a joke, it actually works. You can use this link to create a new site, or you know, this one to create a regular one.

This increase is thanks to the launch of our new Ceph based storage cluster, which we’ve designed to scale and be cost effective in the long term, while still providing the reliability, durability, and performance the modern web comes to expect.

In addition to providing enormous storage capacity for future growth, our serving infrastructure is now completely redundant in the event of equipment failure. If any networking equipment or servers go down, a backup automatically kicks in, allowing Neocities sites to continue being served and updated even if 2/3rds of our servers go down, or if we lose half of our upstream internet connections.

With this launch, and our recently deployed 11-datacenter global anycast caching CDN, Neocities becomes a world-class web hosting service, proving you can still make building a web site fun and easy without sacrificing performance, and without being forced to figure out how to use esoteric, expensive cloud services.

We’ll be doing some deep dives into our new infrastructure in the future, including discussing how we built our own anycast network, and how careful design allowed us to reduce our bandwidth costs by 18x compared to the leading cloud and hosting providers. Stay tuned for that.

These upgrades were made possible by our supporters: Thank you!

More space, more speed. What are you waiting for? Make your own web site now!

People create amazing sites on Neocities, and we’ve wanted to provide a way for people to give back to the awesome people working on them. So we did!

Introducing Neocities Site Tipping. Now any site on Neocities can enable and start receiving site donations!

If you have a Neocities site and want to get started, Visit your Settings page, click Manage Site Settings and click the Tipping tab. You will need to provide the email address for your PayPal account, which will enable both PayPal and Credit Card donations to be sent.

If you’d like to use a Bitcoin address instead, no problem! Just enter your Bitcoin address, and it will be integrated in your site profile. Bitcoin donations won’t show up on news feeds yet, but we’ll look for ways to improve this in the future.

I’m calling site tipping a beta for now, just in case something goes horribly wrong (abuse issues, etc), but I don’t expect any problems we can’t solve.

It’s important to note that Neocities will not deduct any fees for donations. PayPal will charge a small fee (which is comparable to the rate for Credit Card transactions), but we won’t receive any of it. If you’d like to support Neocities as well, we have a separate donations page, or you can help sustain Neocities by becoming a supporter.

Finally, if you’re feeling tipsy, here’s a list of some of the many sites on Neocities now accepting tips.

Enjoy!

Starting January 1st, 2017, Neocities will be defaulting to SSL on all sites. This could possibly lead to some sites breaking, so it’s important that you read this if your site links to any third party content that’s not on Neocities.

Why are you doing this?

I’m not getting into specifics right now, suffice to say there’s recently been a troubling rise in political extremism around the world. To protect your users, it’s critical that we switch to default encryption and make it harder to violate your user’s privacy.

In addition, search engines have recently said they will start penalizing unencrypted sites in search results, meaning that without default SSL, your site will be less likely to show up on search results. We obviously don’t want that to happen, so that is another large motivator.

What’s going to break?

Basically, anything that embeds content (such as images and videos) from a third-party site using http instead of https. For example, if you have an embedded image with <img src="http://example.com/image.png"> instead of <img src="https://example.com/image.png"> (see the s in http?). The problem is that web browsers will block loading of http content because it is considered Mixed Content. If your browser is expecting an HTTPS connection, it refuses to load any unencrypted HTTP content because it “leaks” the encryption.

The fix for this is to make sure your site does not have any embedded content that uses http instead of https. You can also strip out the protocol part of the link (http:) like this, and it will automatically assume HTTPS: <img src="//example.com/image.png">

Is there any way to automatically fix this?

Most browsers will auto upgrade HTTP connections to HTTPS using a special Upgrade Insecure Requests header we’ll enable on our servers. This will take care of a lot of it for most users. However, if the third-party site does not provide HTTPS in addition to the unencrypted HTTP, it will still break. And unfortunately, there are still some older browsers that don’t yet have support for it. Sadly, this also includes the latest versions of Internet Explorer. That’s why we’ve been waiting before doing this transistion. But given the political environment, I don’t think we can afford to wait to make this change anymore.

Will this include Custom Domains?

Yes. We’re already using Let’s Encrypt for all custom domains, and it should already be working. The only difference is that we will be defaulting to it for all incoming traffic.

Why not just directly fix the code on the sites?

I’ve pondered this, but we’ve never made any changes to other people’s HTML before, and I’m not comfortable changing that policy now. I feel like not modifying our user’s sites is part of our trust model, and I’d really rather not go down that slippery slope.

I apologize in advance for any issues this may bring up. But I hope you understand why it’s important to do this, even if there will be some temporary migration pains associated with it. Thanks for your understanding.

– Kyle

Early this year, the Internet Archive put out a call for a distributed web. We heard them loud and clear.

Today I’m making an announcement that begins our long journey to the future of the web. A web that is faster, more secure, more robust, and more permanent.

Neocities has collaborated with Protocol Labs to become the first major site to implement IPFS in production. Starting today, all Neocities web sites are available for viewing, archiving, and hosting by any IPFS node in the world. When another IPFS node chooses to host a site from Neocities, that version of the site will continue to be available, even if Neocities shuts down or stops hosting it. The more IPFS nodes seed Neocities sites, the more available (and redundant) Neocities sites become. And the less centrally dependent the sites are on us to continue existing.

What is IPFS? From their README:

IPFS is a distributed file system that seeks to connect all computing devices with the same system of files. In some ways, this is similar to the original aims of the Web, but IPFS is actually more similar to a single bittorrent swarm exchanging git objects. IPFS could become a new major subsystem of the internet. If built right, it could complement or replace HTTP. It could complement or replace even more. It sounds crazy. It is crazy.

IPFS is still in the alpha stages of development, so we’re calling this an experiment for now. It hasn’t replaced our existing site storage (yet). Like with any complex new technology, there’s a lot of improvements to make. But IPFS isn’t vaporware, it works right now. You can try it out on your own computer, and already can use it to help us serve and persist Neocities sites.

The message I want to send couldn’t possibly be more audacious: I strongly believe IPFS is the replacement to HTTP (and many other things), and now’s the time to start trying it out. Replacing HTTP sounds crazy. It is crazy! But HTTP is broken, and the craziest thing we could possibly do is continue to use it forever. We need to apply state-of-the-art computer science to the distribution problem, and design a better protocol for the web.

Part 1: What’s wrong with HTTP?

The Hypertext Transfer Protocol (HTTP) has unified the entire world into a single global information protocol, standardizing how we distribute and present information to eachother.

It is inconceivable for me to even think about what life would be like without it. HTTP dropped the cost of publishing content to almost nothing, an innovation that took a sledgehammer to the top-down economic, political, and cultural control over distribution of information (music, ideas, video, news, games, everything). As a result of liquifying information and making it the publication of it more egalitarian and accessible, HTTP has made almost everything about our culture better.

I love HTTP, and I always will. It truly stands among the greatest and most important inventions of all time.

But while HTTP has achieved many things, it’s usefulness as a foundation for the distribution and persistence of the sum of human knowledge isn’t just showing some cracks, it’s crumbling to pieces right in front of us. The way HTTP distributes content is fundamentally flawed, and no amount of performance tuneups or forcing broken CA SSL or whatever are going to fix that. HTTP/2 is a welcome improvement, but it’s a conservative update to a technology that’s beginning to show its age. To have a better future for the web, we need more than a spiced up version of HTTP, we need a new foundation. And per the governance model of cyberspace, that means we need a new protocol. IPFS, I’m strongly hoping, becomes that new protocol.

HTTP is brittle

This is a picture of the first HTTP web server in the world. It was Tim Berners-Lee’s NeXT computer at CERN.

Pasted on the machine is an ominous sticker: “This machine is a server, do not power it down!!”.

The reason it couldn’t be powered down is that web sites on other servers were starting to link to it. Once they linked to it, they then depended on that machine continuing to exist. If the machine was powered down, the links stopped working. If the machine failed or was no longer accessible at the same location, a far worse thing happened: the chain between sites becomes permanently broken, and the ability to access that content is lost forever. That sticker perfectly highlights the biggest problem with HTTP: it erodes.

Tim’s NeXT cube is now a museum piece. The first of millions of future dead web servers.

You’ve seen the result:

Even if you’ve never read the HTTP spec, you probably know what 404 means. It’s the error code used by HTTP to indicate that the site is no longer on the server at that location. Usually you’re not even that lucky. More often, there isn’t even a server there anymore to tell you that the content you’re looking for is gone, and it has no way to help you find it. And unless the Internet Archive backed it up, you’ll never find it again. It becomes lost, forever.

The older a web page is, the more likely it is you’ll see 404 pages. They’re the cold-hearted digital tombstones of a dying web, betraying nothing about what knowledge, beauty, or irreverent stupidity may have once resided there.

One of my favorite sites from the 90s web was Mosh to Yanni, and viewing the site today gives a very strong example of how inadequate HTTP is for maintaining links between sites. All the static content stored with the site still loads, and my modern browser still renders the page (HTML, unlike HTTP, has excellent lasting power). But any links offsite or to dynamically served content are dead. For every weird example like this, there are countless examples of incredibly useful content that have also long since vanished. Whether eroding content is questionable crap or timelessly useful, it’s still our history, and we’re losing it fast.

The reason this happens is simple: centrally managed web servers inevitably shut down. The domain changes ownership, or the company that ran it goes out of business. Or the computer crashes, without having a backup to restore the content with. Having everyone run their own personal HTTP server doesn’t solve this. If anything, it probably makes it worse.

HTTP encourages hypercentralization

The result of this erosion of data has been further dependence on larger, more organized centralized services. Their short-term availability tends to be (mostly) good due to redundant backups. But this still doesn’t address long-term availability, and creates a whole new set of problems.

We’ve come a long way since John Perry Barlow’s A Declaration of the Independence of Cyberspace. As our electronic country becomes more influential and facilitates the world with more information, governments and corporations alike have started to pry into HTTP’s flaws, using them to spy on us, monetize us, and block our access to any content that represents a threat to them, legitimate or otherwise.

The web we were intended to have was decentralized, but the web we have today is very quickly becoming centralized, as billions of users become dependent on a small handful of services.

Regardless of whether you think this is a legitimate tradeoff, this was not how HTTP was intended to be used. Organizations like the NSA (and our future robot overlords) now only have to intercept our communications at a few sources to spy on us. It makes it easy for governments to censor content at their borders by blocking the ability for sites to access these highly centralized resources. It also puts our communications at risk of being interrupted by DDoS attacks.

Distributing the web would make it less malleable by a small handful of powerful organizations, and that improves both our freedom and our independence. It also reduces the risk of the “one giant shutdown” that takes a massive amount of data with it.

HTTP is inefficient

As of this writing, Gangnam Style now has over 2,344,327,696 views. Go ahead, watch it again. I’ll wait for you.

Let’s make some assumptions. The video clocks in at 117 Megabytes. That means (at most) 274,286,340,432 Megabytes, or 274.3 Petabytes of data for the video file alone has been sent since this was published. If we assume a total expense of 1 cent per gigabyte (this would include bandwidth and all of the server costs), $2,742,860 has been spent on distributing this one file so far.

That’s not too bad… if you’re Google. But if you’re a smaller site, the cost to serve this much data would be astronomical, especially when bandwidth rates for small players start around $0.12 per gigabyte and go as high a $0.20 in Asia. I’ve spent the better part of my work at Neocities battling expensive bandwidth to ensure we can keep running our infrastructure at low cost.

HTTP lowered the price of publishing, but it still costs money, and these costs can really add up. Distributing this much data from central datacenters is potentially very expensive if not done at economies of scale.

What if, instead of always serving this content from datacenters, we could turn every computer on an ISP’s network into a streaming CDN? With a video as popular as Gangnam Style, it could even be completely downloaded from within an ISP’s network, not requiring numerous hops over the internet backbone. This is one of the many things IPFS is capable of improving (we’ll discuss this in a bit).

HTTP creates overdependence on the Internet backbone

When content is hypercentralized, it makes us highly dependent on the internet backbones to the datacenters functioning. Aside from making it easy for governments to block and censor content, there are also reliability problems. Even with redundancies, major backbones sometimes get damaged, or routing tables go haywire, and the consequences can be drastic.

I got a weird taste of that a few months ago, when Neocities slowed down after a car crashed into a fiber uplink we use in Canada (no suspects yet, but a few promising leads). I’ve also heard stories where hunters have shot at the fiber cables connecting the eastern Oregon datacenters (the enormous ones that store a lot of data), requiring engineers to show up on snowmobiles with cross country skis to repair the fiber lines. Since I wrote this post, details have emerged on a sophisticated attack on fiber lines happening in the Bay Area. The point is, the internet backbone isn’t perfect, it’s easy to attack it, and it’s easy for service to get affected by a few important fiber lines getting cut.

Part 2: How IPFS solves these problems

We’ve discussed HTTP’s problems (and the problems of hypercentralization). Now let’s talk about how IPFS, and how it can help improve the web.

IPFS fundamentally changes the way we look for things, and this is it’s key feature. With HTTP, you search for locations. With IPFS, you search for content.

Let me show you an example. This is a file on a server I run: https://yorniq.xyz/img/neocitieslogo.svg. Your browser first finds the location (IP address) of the server, then asks my server for the file using the path name. With that design, only the owner (me) can determine that this is the file you’re looking for, and you are forced to trust that I don’t change it on you by moving the file, or shutting the server down.

Instead of looking for a centrally-controlled location and asking it what it thinks /img/neocitieslogo.svg is, what if we instead asked a distributed network of millions of computers not for the name of a file, but for the content that is supposed to be in the file?

This is precisely what IPFS does.

When neocitieslogo.svg is added to my IPFS node, it gets a new name: QmXGTaGWTT1uUtfSb2sBAvArMEVLK4rQEcQg5bv7wwdzwU. That name is actually a cryptographic hash, which has been computed from the contents of that file. That hash is guaranteed by cryptography to always only represent the contents of that file. If I change that file by even one bit, the hash will become something completely different.

When I ask the IPFS distributed network for that hash, it efficiently (20 hops for a network of 10,000,000) finds the nodes that have the data using a Distributed Hash Table, retrieves it, and verifies using the hash that it’s the correct data. Early DHT designs had issues with Sybil attacks, but we have new ways to address them, and I’m very confident this is a solvable problem (unlike the problems with HTTP, which are just going to be broken forever).

IPFS is general purpose, and has little in the way of storage limitations. It can serve files that are large or small. It automatically breaks up larger files into smaller chunks, allowing IPFS nodes to download (or stream) files from not just one server like with HTTP, but hundreds of them simultaneously. The IPFS network becomes a finely-grained, trustless, distributed, easily federated Content Delivery Network (CDN). This is useful for pretty much everything involving data: images, video streaming, distributed databases, entire operating systems, blockchains, backups of 8 inch floppy disks, and most important for us, static web sites.

IPFS files can also be special IPFS directory objects, which allow you to use human readable filenames (which transparently link to other IPFS hashes). You can load the directory’s index.html by default, the same way a standard HTTP server does. Using directory objects, IPFS allows you to make static web sites exactly the same way you make them today. It’s a single command to add your web site to an IPFS node: ipfs add -r yoursitedirectory. After that, it’s available from any IPFS node without requiring you to link to any hashes in the HTML (example, and example with index.html renamed).

Federating data with IPFS

IPFS doesn’t require every node to store all of the content that has ever been published to IPFS. Instead, you choose what data you want to help persist. Think of it like bookmarks, except instead of bookmarking a link to a site that will eventually fail, you back up the entire site for yourself, and volunteer to help to serve the content to others that want to look at it.

If a lot of nodes host a little bit, these little bits quickly add up to more space, bandwidth and availablity than any centralized HTTP service could ever possibly provide. The distributed web will quickly become the fastest, most available, and largest store of data on the planet earth. And nobody will have the power to “burn books” by turning it all off. This Library of Alexandria is never going to burn down.

Copying, storing and helping serve web sites from other IPFS nodes is easy. It just takes a single command and the hash of the site: ipfs pin add -r QmcKi2ae3uGb1kBg1yBpsuwoVqfmcByNdMiZ2pukxyLWD8. IPFS takes care of the rest.

IPNS

IPFS hashes represent immutable data, which means they cannot be changed without the hash being different. This is a good thing because it encourages data persistence, but we still need a way to find the latest IPFS hash representing your site. IPFS accomplishes this using a special feature called IPNS.

IPNS allows you to use a private key to sign a reference to the IPFS hash representing the latest version of your site using a public key hash (pubkeyhash for short). If you’ve used Bitcoin before, you’re familiar with this - a Bitcoin address is also a pubkeyhash. With our Neocities IPFS node, I signed the image of Penelope (our site mascot) and you can load it using our IPNS pubkeyhash for that node: QmTodvhq9CUS9hH8rirt4YmihxJKZ5tYez8PtDmpWrVMKP.

IPNS isn’t done yet, so if that link doesn’t work, don’t fret. Just know that I will be able to change what that pubkeyhash points to, but the pubkeyhash will always remain the same. When it’s done, it will solve the site updating problem.

Now we just need to make the location of these sites human-readable, and we’ve got all the pieces we need.

Human-readable mutable addressing

IPFS/IPNS hashes are big, ugly strings that aren’t easy to memorize. So IPFS allows you to use the existing Domain Name System (DNS) to provide human-readable links to IPFS/IPNS content. It does this by allowing you to insert the hash into a TXT record on your nameserver (if you have a command line handy, run this: dig TXT ipfs.git.sexy). You can see this in action by visiting http://ipfs.io/ipns/ipfs.git.sexy/.

Going forward, IPFS has plans to also support Namecoin, which could theoretically be used to create a completely decentralized, distributed web that has no requirements for a central authority in the entire chain. No ICANN, no central servers, no politics, no expensive certificate “authorities”, and no choke points. It sounds crazy. It is crazy. And yet, it’s completely possible with today’s technology!

IPFS HTTP gateway: The bridge between the old web and the new

The IPFS implementation ships with an HTTP gateway I’ve been using to show examples, allowing current web browsers to access IPFS until the browsers implement IPFS directly (too early? I don’t care). With the IPFS HTTP gateway (and a little nginx shoe polish), we don’t have to wait. We can soon start switching over to IPFS for storing, distributing, and serving web sites.

How we’re using IPFS now

Our initial implementation of IPFS is experimental and modest, for now. Neocities will be publishing an IPFS hash once per day when sites are updated, accessible from every site profile. This hash will point to the latest version of the site, and be accessible via our IPFS HTTP gateway. Because the IPFS hash changes for each update, this also enables us to provide an archive history for all the sites, something we automagically just get from the way that IPFS works anyways.

How we’ll use IPNS in the future

Long-term, if things go well, we want to use IPFS for storing all of our sites, and issue IPNS keys for each site. This would enable users to publish content to their site independently of us. If we do it right, even if Neocities doesn’t exist anymore, our users can still update their sites. We effectively take our user’s central dependence on our servers and smash it to pieces, permanently ruining our plans for centralized world domination forever. It sounds awesome. It is awesome!

It’s still early, and there’s much work to do before IPFS can replace HTTP without needing to describe the idea as crazy. But there’s no time like the present to plan for the future. It’s time for us to get to work. Accept the Internet Archive’s challenge: distribute the web.

– Kyle