Web Mentions

One of my favourite features of hosted blogging platforms, such as Medium, is the simple feedback systems they have in place. Medium posts display a number of claps and comments. Each user can give a post up-to 20 claps based on how much they like a post. This means that, at a glance, you have information that can tell you how valuable the content of a post is considered. Sites like Reddit and Hacker News use an upvote/downvote system for a similar purpose and most social media sites have some form of "likes". Unfortunately, like many other software development bloggers, I self host my blog.,,

I could create my own like system but I am very aware that any such system would require some form of account. Most people don't sign up for accounts on random blogs and so the like system would be unlikely to get much, if any, interaction. The aim here is to try and show specific posts as well recieved so I need something that, much like my comments (which use utteranc.es and Github), can either make use of an account/platform that a user is likely to already have, or doesn't require an account at all.

Enter Webmentions.

Webmentions is a 2017 W3C recommendation that describes a simple protocol via which a site can be notified about any links or interactions posted about it. This means that if I were to post a tweet about my latest blog post, and twitter supported webmentions, that I could show the number of likes the tweet recieved on the blog as well. Without having to manually link the tweets to my posts via the twitter api.

Unfortunately, Twitter doesn't actually support web mentions; Nor does Reddit, Github or Facebook. Thankfully there is a work around. Bridgy is a non-profit application that allows you to use the webmention system with these platforms that currently lack a native implementation.

Building a webmentions endpoint

First things first, we need to create an endpoint for webmentions. If you run a static blog, then the third party platform webmentions.io can be used instead of a custom endpoint as it provides a JSON API feed that you can pull in at build time and avoid any of the complexity I'm about to go into. I personally don't want to have to make constant requests to a third party though so I'll be building this from scratch.

The webmentions spec indicates that your endpoint should take a POST request containing the following two parameters.

• source
• target

If either of these paramiters is missing, or they have the same value, then you should reject the request. target should be a url under your control, if it isn't you can again reject the request. target may contain a fragment identifier (a hash symbol followed by the ID of an element on the page that a bowser can scroll to upon loading), for the purposes of the two previous checks, this part of the url should be ignored.

Once the URL's have been validated, you should fetch the html from the source page and parse the microformats2 data so you can work out the mention type and details.

To avoid turning your site into a potential DDOS attack tool, you must process the webmention notifications asynchronously. This is the main missing feature of existing nodeJS implementations. To do this, you should store the notifications in a queue (preferably in a database of some kind to avoid data loss) and process them on some sort of trigger (such as a cron task) as well as removing any duplicates.

To handle this I ended up putting together a nodeJS library called webmention-handler because I found that there wasn't one that completely fit the spec and writing it all out as a tutorial would genuinly have been a 30 odd minute read.

Using my library, I created a MongoDB Storage Implementation (which I will release at a future point) and then created an instance of the main Web Mention Handler that would use it. The docs include a breakdown of how to write such a storage implementation as well as an example of an in memory implementation which you really shouldn't use for anything but testing.

import { WebMentionHandler, type WebMentionOptions } from "webmention-handler";
import MongoWebMentionStorageHandler from 'path/to/your/storage/handler';

const options:WebMentionOptions = {
  storageHandler: MongoWebMentionStorageHandler,
  supportedHosts: ['mikevdv.dev'],
  requiredProtocol: 'https',
  stripQueryParameters: true
};

// Create a new instance of the web mention handler passing in our options
export const webmentionHandler = new WebMentionHandler(options);

As you can see, I've enforced a requirement to use https and am stripping query paramiters from the target url. I did this as I often shared links to twitter with a cachebuster on the url (?x=1) so I could update the OG Image. The options you don't see used above include the whitelist/blacklist functionality which works much like the supportedHosts field. The supportedHosts field allows you to create a single WebMention endpoint that supports multiple sites, but in this case, we only support this one.

From there, I can import the webmention Handler instance to our endpoint and pass in the required fields listed at the top of this section. I'm going to do this as a sveltekit endpoint but you can do much the same thing in whatever framework you like. I include an example Express set up in the package docs.

/* webmentions.ts */
import { webmentionHandler } from "path/to/web/mention/handler";

/** @type {import('@sveltejs/kit').RequestHandler} */
export const post = async ({ request }) => {
  // Get the posted source and target
  const body: FormData = await request.formData()
  const source = body.get('source') as string;
  const target = body.get('target') as string;
  
  // The webmentionHandler does take care of a lot of the validation
  // but I'd recomend cleaning them up more to avoid any issues

  try{
    const res = await webmentionHandler.addPendingMention(source, target);
    return {
      status: res.code,
      body: 'recieved'
    };
  } catch(e) {
    // This isn't perfect as currently, internal errors will be treated as if
    // the user made a mistake. but that's an improvement I still would like to make.
    return {
      status: 400,
      body: e.message
    }
  }
  
}

With that, the endpoint is set up. and we can start recieving web mention notifications. We do however still need to process the notifications as well as tell sites that we support web mentions by adding our endpoint to the head of our pages.

<head>
  <link href="https://mikevdv.dev/webmentions" rel="webmention" />
</head>

Processing Web Mentions

To process our webmentions, we can take a couple of aproaches. If you have a site hosted on a VPS or a EC2 instance, you can write a script and exicute it via a cronjob. If you want to keep everything in a single node application you can use setInterval or you can take the same approach that I did and stick it on a cloud function.

For the sake of security, I personally would avoid running this step on your priamry server. Especially if you are not using the whitelist functionality. As this step sends requests to other servers based on third party notifications, as such it has the potential to leak your server's IP address to the outer web.

Obviously if you are using the example in-memory storage implementation, then there will be issues with having 2 seperate applications as they won't both have access to the storage instance.

The actual script is fairly straight forward, we can import the same handler that we created before. We then need to call its processPendingMentions function. I wrap it in the relevant cloud function wrapper as my cloud function is triggered by a cloud event (in this case time based execution via PubSub and Cloud Scheduler).

import { cloudEvent } from '@google-cloud/functions-framework';
import { webmentionHandler } from "path/to/web/mention/handler";

export const handleWebMentions = cloudEvent('handleWebMentions', async () => {
  await webMentionHandler.processPendingMentions();
});

The processPendingMentions function will fetch pending mentions from your storage implementation, run any further checks that are needed and fetch the html from the provided source. Once it has the html, it will parse it for any supported mentions of your page and store them in whatever storage you've set up.

Using Web Mentions

Once you have some webmentions. How you use them is really up to you. mikevdv.dev is currently only using likes which you might see if you scroll to the bottom of this article. Whatever you choose to do with them, fetching specific web mention types from storage is made really simple by webmention-handler.

const likes = await webmentionHandler.getMentionsForPage(post.url, 'like');
const replies = await webmentionHandler.getMentionsForPage(post.url, 'reply');

webmention-handler won't take care of potential cross site scripting issues so, as ever, displaying third party content should be done with care.

Wrapping up

I have yet to implement sending web mentions from mikevdv.dev but there are a few libraries for doing that which I will have a look into once I get my birthday celebrations out the way. I think this is a really powerful standard and wish more sites would support it natively. It's really suprising to me that there wasn't a library for integrating web mentions in a nodeJS site already (other than webmention-verifier which unfortunately doesn't take the whole DDOS issue into account).

Here's hoping that by creating this library, I can encourage others to implement it.

Edit: Also for the sake of example, here is a tweet of this article that when liked, your like will show up in the likes section of this page (after about an hour).