Embedding content with oEmbed
May 27 2020 · Web Technologies
Embedding external content has been a feature of the web since the introduction of the iframes ages ago. However, embedding as a business strategy didn’t seem to be a thing until sometime around the late 2000s or the early 2010s, as social networking became big business, blogging became really popular, and there was concern over walled gardens. In this environment, embedding became a component for growth, and it was no doubt successful for now behemoths like Youtube and Twitter (another component was adding social networking to sites, à la Google+ or Dunder Mifflin Infinity). Almost any site dealing in content had an embed feature or an embedded “widget.” Even Grovo, where I worked, was on this train as well with the Grovo Widget, though I was not involved in its development. It some cases this made sense and provided utility, in many others it was just copying what seemed to be working for others in ecosystem without regard for product or overall business strategy.
It seems like around this time oEmbed was drafted and Embedly was founded.
My view on how embeds were done was based on what you see in most UIs: you get a embed code (which is a snippet of HTML, likely encapsulated within an iframe), you paste that into a page, and the browser does the rest.
I learned about oEmbed working on Mural. oEmbed is an interesting protocol which allows consumers to request data representing what a resource should look like in an embedded context, given the URL of that resource. The overall flow looks something like this:
Figuring out what the oEmbed endpoint is involves looking at a <link> element from the resource’s HTML page. Unfortunately this does mean you have the overhead and complexity of downloading and parsing an HTML document to get the endpoint. An alternative is pulling from the list of providers in the oEmbed repo and looking at the oEmbed endpoints and allowed URL schemes for resources.
In either case, the result for the end-user is that by simply providing the URL for a resource, the appropriate content for an embed can be provided. Here’s what this looks like in Squarespace:
Unfortunately, the video above is a lie. While everything described around oEmbed would allow for a flow like that, Squarespace (and a surprising number of other popular sites, like Medium) outsource handling of oEmbed to Embedly.
Embedly seems to do a few other things, but primarily it seems to proxy oEmbed content. What’s the value-add? According to Embedly:
We take care of every step of the process: retrieving information about a URL, checking it against malware registries, extracting content, making additional API calls to providers that support them, parsing RSS feeds, and performing validation. We save you time so that you can focus on making your app great.
The only aspect there I find compelling for the price tag is “checking it against malware registries,” but there’s little info on what level of protection they’re actually providing there.
I dealt with Embedly directly when working on Mural and it was a frustrating experience. First, note that if you’re not registered as an Embedly provider, sites that proxy through Embedly will return incorrect oEmbed data. On Squarespace, I noticed the Embedly response would have a type of “link” and not provide any of the data to do an HTML embed (so what’s shown in the video above will not happen, and it will appear as if the resource is not embeddable). Registering as a provider involves filling out a form with some endpoint information and example URLs, easy enough, but I had to wait weeks with no responses or status updates. For a request put in on Jan. 31, 2018, the Embedly integration was not done until Apr. 9th, 2018. A terrible experience overall and surprising for a company that is (a) owned by Medium and (b) seems to be a critical dependency for so many sites.
The short of it is, if as a provider, users can’t embed your content on a site, see if you need to register as an Embedly provider. If you do, good luck.
The excitement and prominence around blogging seems to have died down and this seems to have corresponded with the excitement around embedding content diminishing as well. That said, I think being able to embed content is still, and will continue to be, a powerful mechanism on the web. Even with its flaws oEmbed works nicely in this landscape. I can’t say the same for Embedly.