Hello, Goblins!

Hello and welcome to a brand new blog. I have lately been experimenting with Spritely Goblins, a new tool for distributed programming written in Scheme¹.

The goal of this blog is to document my process of experimentation, discovery, possible frustrations, and exciting moments using this framework to make interesting things! Hopefully these experiences will help other new folks navigate the fascinating world of Goblins, and perhaps along the way we can make things easier for new users as well.

But for today, we will explore a small game in Goblins!

Prerequisites

So, I have already spent some time researching and learning about Goblins. The following are great resources to understand the basics of what the heck is even going on or why I would even want to use it! I recommend checking out at least one of these if you have no idea what Goblins even is.

• For visual learners, you may want to watch this great, quick video tour of Goblins and what it’s like to code with it.
• Perhaps you love lisp and just diving right into the code is the best way. In that case, head on over to core.scm where there are a ton of detailed comments about how things work in the Goblins Code Repository.
• The official Goblins manual for Guile is quite nice, and detailed. Check it out here.
• Or, for those who like everything to be laid out in text, in one long detailed document, check out The Heart of Spritely, a paper that goes over all the concepts in great detail (but it remains quite a fun read!)

Installing

How should you install? Actually, this was a bit tricky for me. But that might just be because I’m running Gentoo. And I’m new to guix. And Emacs! I’m… pretty new to all of this. But that’s okay, I love to learn.

But I will keep that until another blog post–because to be quite honest, it was rather boring and tedious to suffer through. For now, just watch me futz around with code!

Gib, Gab, Gob

Time to dive right in and get coding! My first project² was to create a decentralized Tic Tac Toe game in Goblins! I shall call it… Gib Gab Gob. Because, goblins, I guess.

But wait, if the game is decentralized, that means there’s no authority. So, how do we trust that both parties are acting fairly? Well, let’s ponder but a moment and come to the conclusion that this is already a common quandary in real life, is it not? In order to determine who goes first, the obvious choice is–you guessed it…

Rock, Paper, Scissors!

That’s right, we took a detour from Tic Tac Toe to Rock Paper Scissors. Oh well, that’s perfectly fine because perhaps Tic Tac Toe, as simple as it is, is too complicated to start off with. And what could be simpler than Rock, Paper, Scissors?

(define-module (gib-gab-gob rps)
  #:use-module (ice-9 match) ;; we'll need this very soon!
  #:use-module (goblins) ;; the titular goblins library
  #:use-module (goblins ocapn captp) ;; network comms
  #:use-module (goblins ocapn ids) ;; how to address network locations
  #:use-module (goblins ocapn netlayer testuds) ;; our method of network comms
  #:use-module (goblins actor-lib methods) ;; methods for actors!
  #:use-module (goblins actor-lib sealers) ;; ways to hide and reveal a value.
  #:use-module (oop goops)) ;; for object oriented programming!

The logic

How to start, how to start. Of course! let’s first throw together the core logic of Rock, Paper, Scissors (let’s call it RPS from now on).

First, our choices:

(define rock-paper-scissors (list 'rock 'paper 'scissors))

Now, let’s define a function that will handle all the possible cases for us, with some quick error handling so we don’t find ourselves in a nasty infinite loop given bad input!

(define (rps-winner a b)
  ;; Checking if both items are members of our rock, paper, scissors trinary.
  (if (and (memq a rock-paper-scissors) (memq b rock-paper-scissors))
      (match (list a b)
	[(x x) 'tie]
	[('rock 'scissors) #t]
	[('rock 'paper) #f]
	[('scissors 'paper) #t]
	[(x y) (not (rps-winner y x))])
      (error "Unexpected item in the shooting area" a b)))

And finally, we shall do some helpful picking of a random choice³:

(define (pick-rps)
  (list-ref rock-paper-scissors (random (length rock-paper-scissors))))

Host and Client

We are going to create a host and a client. The reason we need to differentiate is merely because someone has to at least initiate the game. But otherwise, we aren’t going to assume authority! Now let’s go over the duties of the two peers.

Host’s duty

• Create a vat, inside which actors will live and do things.
• Register a lobby actor on the network.
• Print its address so we can give it to the client, to allow connecting.
• Some sort of … game? We’ll concern ourselves with that later.

Client’s duty

• Create a vat as well.
• Find the lobby actor on the network, given the address provided by the host.
• Join the lobby, by calling a method on the lobby actor.
• Participate in aforementioned bizarre ritual.

Our first actor

Let’s create this Lobby actor so we can start our morbid goblin ritual. By convention, Goblin actors have a little hat (^). This lobby will have a single method, register-opponent, which the client will invoke.

(define (^game-lobby bcom)
  (define pick (pick-rps))
  (methods
   ;; Let's fill out the logic for this method later; for now let's just say hi.
   [(register-opponent name) (format #t "Hello, ~s!" name)]))

Host logic

Oop Goops!?

Sounds… slimy, somehow. But actually it’s just Guile’s official Object Oriented Programming library. Why am I using it here? Well, the Community Garden project did (for a better reason), and I am new to it, so I figured I would give it a shot and create classes for both a host and a client. I think in a lot of cases you might not really need to use this, and instead just use Goblin’s actors directly (perhaps?). Regardless, the library is documented decently elsewhere. So, I’m going to gloss over the specific details of GOOPS.

Defining the Host

Here we are creating a host class, with some accessors and initializing some things. We also define an initialize method for the rps-host class, which will automatically be called when the class is instantiated. Finally we reach some Goblins specific stuff! I have annotated what we are doing in order to better explain.

(define-class <rps-host> ()
  ;; We spawn a vat here, when the object is instantiated! Poof!
  (vat #:accessor vat #:init-thunk spawn-vat)
  (lobby #:accessor lobby)
  (user-name #:accessor user-name #:init-keyword #:user-name))

(define-method (initialize (host <rps-host>) initargs)
  ;; This line handles the class initializations defined above so properties are defined!
  ;; Before I added this, I was a bit confused at first because I got a strange error,
  ;; but it turned out to be from using uninitialized object properties.
  (next-method)
  ;; Let's use this shiny, freshly cleaned and seasoned cast iron vat!
  ;; Everything involving Actors must take place inside a vat, so might as well use this one!
  (with-vat
   (vat host)
   ;; First, let's spawn our ^game-lobby actor and set it as a property on the object.
   (set! (lobby host) (spawn ^game-lobby))
   ;; A netlayer is simply the means by which we are communicating over the net.
   ;; For now, we are just using Unix sockets because that makes testing easier.
   ;; We create the netlayer with a helper method I have so far omitted but will soon reveal!
   ;; We also spawn a mycapn instance! This is the network coordinator, so to speak.
   (define mycapn (spawn-mycapn (new-testuds-netlayer)))
   ;; sref? That's short for `sturdyref`. It's a reference to our Actor that anyone on
   ;; the internet can use to find us. Cool!
   ;; Whoa! what's that dollar sign??? If you read the prereqs, you should understand
   ;; that this means that we are invoking the `'register` method on the `mycapn` Actor,
   ;; within the same vat as mycapn is living (so it can be done synchronously, right away!).
   (define lobby-sref ($ mycapn 'register (lobby host) 'testuds))
   ;; Now, let's turn our sturdyref into a string, and print it!
   (format #t "Connect to: ~a\n" (ocapn-id->string lobby-sref))))

Quick detour to make some helper code

We are using UDS (unix domain sockets) for this, because TOR is slow, and that’s the only way to properly communicate over the network in Goblins right now. There’s a testuds netlayer, but let’s make a helper to make it easier for ourselves.

(define (new-testuds-netlayer)
  (define tmp "/tmp/netlayers")
  ;; We are simply creating a folder if it doesn't exist
  (unless (access? tmp X_OK) (mkdir tmp))
  ;; And then spawning the netlayer actor!
  (spawn ^testuds-netlayer tmp))

The Host is Ready!

We can now simply evaluate (make <rps-host> #:user-name "alice") to create and print our address to the lobby!

Client logic

Not bad at all. We now have a host, so let’s connect to it with a client.

(define-class <rps-client> ()
  (vat #:accessor vat #:init-thunk spawn-vat)
  (user-name #:accessor user-name #:init-keyword #:user-name)
  (addr #:accessor addr #:init-keyword #:addr)
  (lobby #:accessor lobby))

Just as before, we spawn our vat. Our client has a few straightforward properties. We have the vat, the username of the client, the address, and the actual lobby for when it is created. Let’s do our initialization as well:

(define-method (initialize (client <rps-client>) initargs)
  (next-method)
  (with-vat
   (vat client)
   ;; In an inverse to before, we are turning the address string into a sturdyref.
   (define lobby-sref (string->ocapn-id (addr client)))
   ;; Creating our mycapn instance!
   (define mycapn (spawn-mycapn (new-testuds-netlayer)))
   (set! (lobby client)
	 ;; "enliven" is a fancy way of saying that we turned the reference into
	 ;; something we can talk to!
	 ($ mycapn 'enliven lobby-sref))))

Hello Lobby!

Oh boy! We can finally register ourselves with the lobby and do some fancy remote procedure calling!

(define (join-rps user-name addr)
  ;; Let's make the client!
  (define client (make <rps-client> #:user-name user-name #:addr addr))
  ;; Doing some functionality with the client's vat now.
  (with-vat
   (vat client)
   ;; Here we finally communicate with the remote vat! We are invoking the method
   ;; 'register-opponent with the user-name argument, and then waiting for it to finish
   ;; before printing a message.
   (on (<- (lobby client) 'register-opponent user-name))
   ;; I kept missing the `_` argument here, which resulted in a strange error.
   ;; Don't forget to add it!
   ;; I believe that my reasoning was coming from JavaScript where you can just
   ;; elide lambda vars.
   (lambda (_)
     (format #t "Ok! we are registered.\n")))))

If we have the host running as mentioned above, we can connect to alice’s lobby with (join-rps "bob" "ocapn://gobbledeegook...")–this is the address printed out by alice.

So, what happens?

Hello, bob!
Ok! we are registered.

Whoa. We just communicated over the “network”. Well, just over a local operating system socket. But if we had used the onion netlayer with the tor daemon as described here, it would have worked over Tor!

Implementing the game flow

So now that bob and alice have introduced themselves, it’s time to add the RPS logic we have been desperate to observe, for so long. And now we run into the first problem with network programming–it’s not really possible to have a synchronized event that happens simultaneously for both users. More concretely, when you play RPS with someone, the idea behind it is that you synchronize your “shot” by chanting “rock, paper scissors, shoot!” or some regional variant. If someone is late to the ‘shot’, they are chastised and immediately shunned from the friend group, a pariah to society. Well, not really. But, at least, they might be accused of being a cheater. Because, they might have a chance to peek at the choice their opponent is making.

But in network land, we can’t really do that. On a traditional website or server-based online game, you would trust a third party, the server, to coordinate the choices of both players, and then dictate which one is the winner! But that’s… authoritarian, and counter to our goal of decentralization.

Rock, Paper, Scissors by mail?

So, perhaps a good way to frame our challenge in the “real world” is that, we want to play our little game using parcels sent with the post office. What if bob were to put faer choice into a secure lockbox⁴, put a few stamps and an address on said lockbox, and send it straight to alice in the mail?

Then, alice, upon receiving the lockbox, knows that zee has bob’s choice, and so can send zer choice to bob, just in a plain insecure envelope, because bob can immediately know the answer as fae has made the choice and cannot change it.

Finally, when bob gets this envelope, fae can send the combination for the lockbox containing faer answer, for alice to open.

Translating to Goblinspeak

So, how do we do this inside Goblins? Well, technically we can’t yet. As yet, Goblins lacks the ability to encrypt an actor, pass it over the network, and then later decrypt it. But we can pretend, with a feature called Sealers! And someday in a future version of Goblins, this will actually be unexploitable by either party⁵.

As a reminder, bob is the client, and alice is the host.

Anyway, just like putting it into a lockbox, we will seal our choice using a Sealer, which produces a new Actor that is just the sealed choice. bob can hold onto the unsealer, which represents the combination or key used to unlock the sealed choice. And we can perform our ordered operations as described with our post-office metaphor:

• bob makes a choice.
• bob seals the choice.
• bob sends alice the sealed Actor, along with a return Actor so that alice has a way to contact faer to give zer pick.
• alice makes a choice.
• alice receives the Actor and sends zer choice to bob.
• bob receives alice’s choice and exchanges it with faer unsealer (at this point, bob knows if zee has won).
• alice gets the unsealer and unseals the sealed choice (at this point both know who has won!).

Apparently, this is a concept someone has written a paper about. I wasn’t aware of this at the time, but perhaps this Lockstep Protocol paper is worth reading for a more formalized explanation of this process and how it relates to decentralized game playing.

The Return Actor

bob needs an Actor for alice to use. We are calling this ^client-picker, which is a name I don’t really like. But I’m new to naming Actors so it’ll have to suffice.

(define (^client-picker bcom)
  ;; spawn-sealer-triplet is a function that gives us three Actors:
  ;; - The seal actor which can seal something.
  ;; - The unseal actor which can unseal something.
  ;; - The 'brand check predicate' which verifies that a sealed Actor
  ;; was sealed by us.
  (define-values (seal-pick unseal-pick my-pick?)
    (spawn-sealer-triplet))
  ;; Choose randomly what we want--remember that function from way back?
  (define pick (pick-rps))
  (methods
   ;; A way to grab a freshly sealed pick from the Actor.
   [(get-sealed-pick) ($ seal-pick pick)]
   ;; A method for client to call to exchange their (not sealed) pick with their unsealer
   [(pick->unsealer peer-pick)
    (format #t "Peer picked ~a... a bold choice (do I win? ~s), i will send my unsealer\n" peer-pick (rps-winner pick peer-pick))
    unseal-pick]))

Modifying the client object

Now, we shall modify join-rps such that it implements all the logic we need bob to perform!

(define (join-rps user-name addr)
  ;; As before.
  (define client (make <rps-client> #:user-name user-name #:addr addr))
  (with-vat
   (vat client)
   ;; Spawn the new picker object.
   (define client-picker (spawn ^client-picker))
   ;; The core logic is implemented with a bunch of chaining,
   ;; which is pretty straightforward.
   ;; Let's go through the order of operations, from the inside
   ;; outwards, because this can be confusing to lisp newbies!
   ;; - Get a sealed pick from the client-picker Actor,
   ;; synchronously with `$`.
   ;; - Remotely invoke 'register-opponent on the lobby Actor,
   ;; asynchronously with `<-`, and wait for it to respond,
   ;; using `on`.
   ;; - Print a message when all done.
   (on (<- (lobby client)
	   'register-opponent
	   ;; Give peer our user name.
	   user-name
	   ;; Give peer our picker to respond to.
	   client-picker
	   ;; And give peer the sealed pick.
	   ;; They could get it themselves, but this reduces round trips.
	   ($ client-picker 'get-sealed-pick))
       (lambda (_)
	 (format #t "~s finished the game.\n" user-name)))))

The Updated Lobby Code

Finally, let’s allow alice to respond to this logic.

(define (^game-lobby bcom)
  (define pick (pick-rps))
  (methods
   [(register-opponent name client sealed-pick)
    ;; Instead of just saying hi (well, they don't actually say hi over the network, but we can imagine), we will
    ;; exchange our pick with their unsealer.
    (format #t "Hey there, ~a! You sent me your pick of rock-paper-scissors; now I will send mine.\n" name)
    ;; We invoke the pick->unsealer conversion method to get the pick, and then apply the unsealer to the sealed pick.
    ;; Note the nested `<-`! That's new. Instead of two layers of lambdas, we can chain <- together.
    ;; Elegant on the code side, but actually, this is even cooler than that. Let's get out of this comment to discuss.
    (on (<- (<- client 'pick->unsealer pick) sealed-pick)
	(lambda (peer-pick)
	  (format #t "opponent ~s has picked ~a (do I win? ~s)\n" name peer-pick (rps-winner pick peer-pick))))]))

Time travel???

Goblins likes to talk a lot about time travel. It’s one of the features the Spritely Institute brags about the most! In fact, it has two different ways to do time travel. But let’s be honest, it’s not really time travel. But it seems like it to the coder and the user, because the magic is done opaquely.

Promise Pipelining

We haven’t really discussed it by name here, but the magic we are doing with <- involves a concept called “promises”. The <- function returns a promise, in fact, and on waits on the promise and invokes the provided lambda with the result of the promise. If you’ve used JavaScript, promises were introduced to modern programmers there a while back. But they actually have their roots in the original inspiration for Goblins, which dates back to the ’90s.

What’s super cool about Goblins (and where “time travel” comes into play) is that these promises are pipelined. This means that if we nest promises without resolving them, Goblins will forward the next request ahead of time to the remote party, so it can return the result with a single response instead of a back and forth. This stuff is truly magical to behold, and I feel that it is best explained in the Cap’n Proto docs (Cap’n Proto is, incidentally, a great way to do RPCs if you still can’t use Goblins itself⁶).

Rollback

The second form of “time travel” is automatic rollback. If anyone encounters an error, the current “churn” of the vats can be rolled back to a previous state, allowing for easy recovery. It’s very easy to add this to most Goblins applications, because of how Goblins programming is structured around the vat churn loop (event loop).

Playing the game

All right. We’ve actually written all the code now. The only thing left is to repeat the function calls we ran before!

scheme@(gib-gab-gob)> (make <rps-host> #:user-name "alice")
scheme@(gib-gab-gob)> Connect to: ocapn://gobblygook[...]
scheme@(gib-gab-gob)> (join-rps "bob" "ocapn://gobblygook[...]")
Hey there, bob! You sent me your pick of rock-paper-scissors; now I will send mine.
Peer picked paper... a bold choice (do I win? #f), I will send my unsealer
bob finished the game.
opponent bob has picked rock (do I win? #t)

And there we have it. Goblins-based Rock, Paper, Scissors.

Conclusion

Well, we haven’t made Tic Tac Toe, but that’s probably enough (or perhaps too much) for a single post. But we’ve covered most of the important concepts in Goblins, actually. We may have even done the hard part.