Writing a Clipboard Manager

Motivation

Clipboard management is very important to my workflow. To me, a clipboard manager is useful in two ways:

1. It extends my (rather poor) temporary mundane memory by caching a few dozens of most recent selections.

2. It synchronizes clipboard and primary selections. Since some programs only support one kind of selection, this is particularly useful.

For the first point, I have to be able to choose from the history by pressing a few keystrokes. Having to touch the mouse during writing sessions is unacceptable. The menu dropping down from the systray is also undesirable because I have a multi-monitor setup. This narrows down to only one plausible option: Diodon, which I having been using on Debian for at least two years. However, as I was migrating to NixOS earlier last month, I was unable to package it for Nix.

Naturally, I went looking for alternatives, most of which I had tried before and did not satisfy my requirements. clipmenu got my attention however: it was made to work with dmenu(-compliant launchers), which I had a rather nice experience with in Sxmo on my PinePhone. However, I use awesome on my workstation and its widget toolkit covers my launcher and menu need perfectly. I don't need dmenu and do not wish to spend time configuring and theming it. Plus, the architecture of dmenu scripts and awesome widgets vastly differs: while awesome executes the input programs, dmenu is called from the scripts.

Inspirations and Design

As even the most plausible candidate is not a suitable replacement, I would need to write my own clipboard manager. clipmenu is not really a good base though because it's written in shell script, something I ain't fluent in.^[1] Its idea is brilliant however:

1. clipmenud uses clipnotify to wait for new clipboard events.

2. If clipmenud detects changes to the clipboard contents, it writes them out to the cache directory and an index using a hash as the filename.

I later translated clipnotify to Zig^[2] and called it clipbuzz.^[3] From clipbuzz's usage,

while clipbuzz
do # something with xclip or xsel
done

and this is exactly how yet another clipboard manager was written, but before we get there, let's talk about this article's sponsor!

I'm kidding d-; though we cannot jump into the implementation just yet: we only resolved the first point out of two. How about the data structure? Hashing sounds like overengineering in this case: nobody needs more than a few dozen entries^[4] and hashes are not very memorable. Printable characters can serve much better as indices.

What? What happens when we run out of them? We reuse/recycle them!^[5] They would also fit within one single line, heck, we just store all of them in order inside a file and rotate each time there's a new selection. Picking would just be moving a char to the beginning. The entire cache directory can just look something like this:

$ ls $XDG_CACHE_HOME/$project
order
R
A
K
U

Wait, is that a sign? We must use Raku to implement $project then… Speaking of $project, I planned to use it with awesome and vicious so let's call it something brutal, like a cutting board, which is thớt in Vietnamese, an Internet slang for thread. Cool, now we have the daemon name, and conventionally the client shall be threac, or threa client.

Daemon Implementation

Reading Inputs

Raku was chosen^[6] for the ease of text manipulation and seamless interfacing with external programs. I learned it quite a while ago and has always been waiting for a chance to do something more practical with it, other than competitive programming which isn't a good fit due to Rakudo's poor performance. In Raku, the snippet from clipbuzz's README becomes:

while run 'clipbuzz' {
    # do something with xclip or xsel
}

Out of all languages I know, this is by far the simplest way to run an external program. Most would require one to import something or do something with the call's return value, and don't even get me start on POSIX fork and exec model.

OK, now what are we gonna do with xclip? One obvious thing would be to read the current selection. Raku got you covered, fam:

my $selection = qx/xclip -out/;

Remember when I said Raku can seamlessly interact with external programs? qx is how you capture their standard output, it is really that simple. But wait, which selection is that? No worries, xclip supports both primary and clipboard:

my $primary = qx/xclip -out -selection primary/;
my $clipbroad = qx/xclip -out -selection clipboard/;

Cache Directory Setup

This is when we write those selection down for later use, right? Well, we need to figure out where to save them first. According to XDG Base Directory Specification, $XDG_CACHE_HOME shall falls back to $HOME/.cache:

my $XDG_CACHE_HOME = %*ENV<XDG_CACHE_HOME> // path $*HOME / '.cache':;

For convenience purposes, I defined the / operator as an alias for path concatination:

multi sub infix:</>($parent, $child) { add $parent: $child }

With $XDG_CACHE_HOME defined, we can prepare the base directory as follows:

my $base = $XDG_CACHE_HOME.IO / 'threa';
mkdir $base: unless $base.e;
die "thread: $base: File exists" when $base.f;

As a wise man once said,

As it often happens, writing an article ends up with a bug found in Rakudo.

In this case, there's a bug in mkdir that makes it happily returns even if the target path is a file. I'm trying to fix it at the moment but a test is still failing. Update: it passed after a maintainer bumped the dependencies to the patched version.

Anyway, back to our clipboard manager. Here we are using flow controllers such as unless and when in the form of statement modifiers, which can sometimes be easier on eyes keeping the code flat. Existence checks like e (exists) and f (file) are also really handy. Next, we check on the order:

constant $ALNUM = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';

sub valid($path) {
    return False unless $path.f;
    so /^\w\w+$/ && .chars == .comb.unique given trim slurp $path
}

my $order = $base / 'order';
spurt $order, $ALNUM unless valid $order;

Instead of printable, we only allow alphanumerics and fallback to the uppercase ones (mainly because my screen can only fit as much vertically), unless $XDG_CACHE_HOME/threa/order is a file, contains at least two unique alphanumerics (exclusively). Reading and writing files in Raku is incredibly trivial, just slurp and spurt the path. Since we are not interested in whitespaces, they are trim'ed from order. Notice that Raku allows subroutines to be called without any parentheses—I love Lisp, but opening parenthesis after the function name always confuses me, especially when nested.

As you might have guessed, given is another statement modifier setting the topic variable that is particularly useful in pointfree programming, where regular expressions (e.g. /^\w\w+$/) are matched against directly and methods are called without specifying the object. Raku is also a weakly-typed language: .comb.unique (a list of unique characters) is coerced into an integer when compared to one (number of .chars).

Comparing and Saving Selections

What do we do with the order then? First we can determine the latest selection and compare it to the ones we got from xclip earlier to see which one is really new. We'll also need to rotate the order, i.e. write the new selection to the $last file and move it in front of the others that we $keep as-is:

my ($first, $keep, $last) = do
    given trim slurp $order { .comb.first, .chop, .substr: *-1 }
my ($other, $content) = do given try slurp $base / $first or '' {
    when * ne $primary { 'clipboard', $primary }
    when * ne $clipboard { 'primary', $clipboard }
}

On the first few run, probably the cache files don't exist just yet, so we fall them back to empty ones using try ... or .... We need to know the $other selection (outdated one) to later synchronize them both. In case of reselection, neither is updated and we simply skip this iteration:

next unless $other;

Otherwise, let's go ahead, write down the $content, rotate $order and synchronize with the $other selection:

my $path = $base / $last;
spurt $path, $content;
spurt $order, $last ~ $keep;
run <xclip -in -selection>, $other, $path

That's it, now put the daemon in $PATH and run it in ~/.xinitrc or something IDK. If you're worried that some selection might be too big to read that you'll the next event, asynchronize the qx calls by prefixing them with start, and await the results later on. It is that easy.

Command-Line Interface

Hol up, what if I want to store the cache elsewhere or use another set of characters? "Then you can go right ahead and have an intercourse with yourself, you ungrateful little piece of [redacted]." I would have said this were I to implement this in other languages, but luckily I got Raku, and Raku got sub MAIN:

sub MAIN(
  :$children where /^\w\w+$/ = $ALNUM, #= alphanumerics
  :$parent = $XDG_CACHE_HOME           #= cache path
) {
    my $snowflakes = $children.comb.unique.join;
    my $base = $parent.IO / 'threa';
    my $order = $base / 'order';

    while run 'clipbuzz' {
        ...
        spurt $order, $snowflakes unless valid $order;
        ...
    }
}

No matter how cool you think this is, it is cooler, I mean, look:

$ thread --help
Usage:
  thread [--children[=Str where { ... }]] [--parent=<Str>]
  
    --children[=Str where { ... }]    alphanumerics
    --parent=<Str>                    cache path

Client Implementation

Back-End

Following the Unix™ philosophy, threac will do only one thing and do it well: it shall take the chosen selection and schedule it to move to the beginning:

my $XDG_CACHE_HOME = %*ENV<XDG_CACHE_HOME> // path add $*HOME: '.cache':;

sub MAIN(
   $choice where /^\w?$/,    #= alphanumeric
  :$parent = $XDG_CACHE_HOME #= cache path
) {
    my $base = $parent.IO.add: 'threa';
    my $order = add $base: 'order';
    spurt $order, S/$choice(.*)/$0$choice/ with $order.slurp;
    my $path = $base.add: $choice;
    run 'xclip', $path
}

The highlight here is the non-destructive substitution S///, which allow regex substitution in a pointfree and pure manner. Though, instead of moving $choice to top of the deque, we place it at the bottom and use xclip to trigger the daemon to do it and synchronize between selections.

Front-End

Note that threac does not give any output: selection history (by default) are stored in a standard and convenient location to be read by any front-end of choice. For awesome I made a menu whose each entry is wired to threac and xdotool (to simulate primary paste with S-Insert) and bind the whole thing to a keyboard shortcut.

local base = os.getenv("HOME") .. "/.cache/threa/"
local command = "threac %s && xdotool key shift+Insert"
local f = io.open(base .. "order")
local order = f:read("*a")
f:close()

local items = {}
for c in order:gmatch(".") do
  local f = io.open(base .. c)
  table.insert(items, {f:read("*a"):gsub("\n", " "), function ()
    awful.spawn.with_shell(command:format(c))
  end})
  f:close()
end
awful.menu{items = items, theme = {width = 911}}:show()

Conclusion

Through writing the clipboard manager threa, which is released under GNU GPLv3+ on SourceHut, we have discovered a few features of Raku that make it a great scripting language:

• Out-of-box CLI support:

  • Running programs and capturing output

  • Environment variables

  • File system operations

  • Builtin argument parser

• Concision:

  • Statement modifiers

  • Topic variable

  • First-class regex

  • Trivial asynchronization

As a generic programming language, Raku has other classes of characteristics that makes it useful in other larger projects such as grammars (i.e. regex on steroids) and OOP for human beings. It is a truly versatile language and I really hope my words can convince someone new to try it out!

[1]	I ain't proud of this, okay?

[2]	I'm obsessed with exotic languages.

[3]	The z's are for Zig, how original, I know.

[4]	[citation needed]

[5]	Wow much environment!

[6]	By some supernatural being of course!

Tags: fun recipe clipboard —Nguyễn Gia Phong, 2021-07-03

Comments

Follow the anchor in an author's name to reply. Please read the rules before commenting.