Archive for September, 2020

Copy & pasting non-textual objects in the browser

Interacting with the clipboard

A proper way to deal with clipboard access has been a dream for web developers for years. There’s out-of-the-box browser support, via keyboard shortcuts and content menu commands, for <input>, <textarea>, or elements with the contenteditable attribute. However, for more complex interactions or dealing with application-defined, non-textual objects (e.g. something composed of multiple DOM elements that is manipulable by the user, such as the ScratchGraph notes and connectors shown below) we need to start looking at the available Javascript APIs.

ScratchGraph entities and connectors

Clipboard API vs document.execCommand() + paste event

The bad news is that document.execCommand() (using the “cut”, “copy” commands) is still the de-facto way of writing to the clipboard, despite this method being deemed obsolete. The good news is that there does seems to be good progress, in terms of stability and browser implementation, of the Clipboard API.

For reading from the clipboard, the paste event seems to be the best way to go, however there is the inherent limitation that this event will only be triggered by “paste actions” from the browser’s interface (e.g. the use hitting Ctrl + V). Again, there is good news in that the Clipboard API would allow more flexibility here and there is good progress towards browser support.

Despite the good news around the Clipboard API, given the state of where things are now, in September 2020, using document.execCommand() and the paste event seems to be the way to go; the Clipboard API, as well as the corresponding permissions via the Permissions API, are still in the process of being implemented in browsers. However, most of what’s in this post will (hopefully) still be valid with the Clipboard API, with the clipboard interaction code being more robust and the more hacky bits being thrown away.

Copying to the clipboard with document.execCommand(“copy”)

Selecting plain text and copying it to the clipboard is straightforward with an <input> or <textarea>:

  • Call the select() method on the element
  • Call document.execCommand("copy");

We can make a generic method to copy arbitrary text to the clipboard by programmatically creating a <textarea>, setting its value to the text we want, performing the above operations to copy its contents to the clipboard, and finally removing the created <textarea>.

const Clipboard = { copyText: function(_text) { const textAreaElem = document.createElement('textarea'); textAreaElem.textContent = _text; document.body.appendChild(textAreaElem); textAreaElem.select(); document.execCommand("copy"); document.body.removeChild(textAreaElem); } };

Now, using this method, if we can serialize an object to some sort of textual format, we can put it on the clipboard. JSON is a good option, as it’s easy to work with in Javascript.

Object โ†’ JSON โ†’ Plain text โ†’ Clipboard

In ScratchGraph, entities like notes, connectors, etc. have a toJSON() method, which returns a JSON serialized representation of the entity, e.g.:

this.toJSON = function() { const serializedObj = { "id": self.getId(), "owner_id": self.ownerId, "sheet_id": self.sheetId, "position_x": self.getX(), "position_y": self.getY(), ... }; return serializedObj; }

When a user initiates a request to copy something, say by hitting Ctrl + C, we iterate over all the selected entities, get the serialized JSON for each, and build an array of JSON objects. Next we construct a JSON object containing that array, along with some metadata around context (application name, version, etc.). Finally, we JSON.stringify this object to get a plain text representation and use the Clipboard.copyText() method to write to the clipboard.

document.addEventListener('keydown', function(e) { // Copy selected entities on Ctrl + C if(e.ctrlKey && e.key === 'c') { const entitiesSelected = currentGroupTransformationContainer.getEntities(); const entitiesJsonArr = []; entitiesSelected.forEach(function(e) { entitiesJsonArr.push(e.toJSON()); }); ... const strForClipboard = JSON.stringify({ "application": "scratchgraph", "version": "1.0", "entities": entitiesJsonArr, ... }); Clipboard.copyText(strForClipboard); } });

Pasting from the clipboard

To read what’s on the clipboard, we can listen for and implement a handler on the paste event.

While you can listen for the paste event on any DOM element, I’ve found it tricky to isolate to specific elements because the element that has focus isn’t always obvious and I’ve run into situations where an offscreen <input> gets focus and the clipboard content simply gets pasted into that input. It’s more reliable to listen on the document and determine if and where to paste something based on the metadata embedded when we copied data to the clipboard.

Sketching out what we need to deal with, we get the following:

  • Listen for paste events on the document
  • Check if there’s plain-text content being pasted
  • Try to parse the plain-text content as JSON
  • Check whatever metadata is in the object to see if it’s something copied from our application and it’s something we can read/interpret.
  • If it’s something we can handle, suppress any default behavior and do what is needed to clone and create new objects.

This is simplified for clarity, but the code in ScratchGraph looks something like this:

document.addEventListener('paste', function(e) { if(typeof e.clipboardData === 'undefined' || typeof e.clipboardData.items === 'undefined') { return; } const items = e.clipboardData.items; for (let i=0; i<items.length; ++i) { if (items[i].kind === 'string' && items[i].type === "text/plain") { try { const clipboardJson = JSON.parse(e.clipboardData.getData('text/plain')); if(clipboardJson.application === "scratchgraph") { e.preventDefault(); createFromClipboardJson(clipboardJson); } } catch(err) { } } } });

The createFromClipboardJson() method handles the application-specific logic of reading the data and creating copies. In ScratchGraph, I’m dealing with entities, so I don’t actually deserialize, I just read the bits of data needed to be make a clone and create something with a new ID (i.e. new entity). However, YMMV, based on the type of objects you’re dealing with, how your application handles data, and/or how you deal with state.

Limitations and future work

As I mentioned, there are limitations here when it comes to reading or writing from/to the clipboard. The paste event will only be triggered by interactions supported by the browser, so creating something like a button to paste content isn’t possible. document.execCommand("copy") is now considered obsolete and the method presented to allow copying arbitrary bits of text is pretty hacky, though it is versatile in that you can bind the method to application-specific interactions (e.g. a button to copy content). A further limitation here is that data can only be copied as plain-text and we’re not actually encoding any type information; this manifests in some non-ideal behavior, where what’s put on the clipboard can be pasted into any application that accepts plain-text.

The Clipboard API looks to be a promising solution to these limitations. I’m hoping to revisit this in the near future to update the clipboard interaction logic to use the API and have an all-round cleaner and more robust solution.