galene-client.md

Writing a new frontend

The frontend is written in JavaScript and is split into two files:

• protocol.js contains the low-level functions that interact with the server;
• galene.js contains the user interface.

A simpler example client can be found in the directory static/example.

A new frontend may either implement Galène's client-server protocol from scratch, or it may use the functionality of protocol.js. This document documents the latter approach.

Data structures

The class ServerConnection encapsulates a connection to the server as well as all the associated streams. Unless your frontend communicates with multiple servers, it will probably create just a single instance of this class.

The class Stream encapsulates a set of related audio and video tracks (for example, an audio track from a microphone and a video track from a webcam). A stream is said to go up when it carries data from the client to the server, and down otherwise. Streams going up are created by the client (your frontend), streams going down are created by the server.

Connecting to the server

First, fetch the .status JSON at the group URL:

let r = await fetch(url + ".status");
if(!r.ok) {
    throw new Error(`${r.status} ${r.statusText}`);
}
let status = await r.json();

Create a ServerConnection and set up all the callbacks:

let sc = new ServerConnection()
serverConnection.onconnected = ...;
serverConnection.onclose = ...;
serverConnection.onusermessage = ...;
serverConnection.onjoined = ...;
serverConnection.onuser = ...;
serverConnection.onchat = ...;
serverConnection.onclearchat = ...;
serverConnection.ondownstream = ...;

The onconnected callback is called when we connect to the server. The onclose callback is called when the socket is closed; all streams will have been closed by the time it is called. The onusermessage callback indicates an application-specific message, either from another user or from the server; the field kind indicates the kind of message.

Once you have joined a group (see below), the remaining callbacks may trigger. The onuser callback is used to indicate that a user has joined or left the current group, or that their attributes have changed; the user's state can be found in the users dictionary. The onchat callback indicates that a chat message has been posted to the group, and onclearchat indicates that the chat history has been cleared. Finally, ondownstream is called when the server pushes a stream to the client; see the section below about streams.

You may now connect to the server:

serverConnection.connect(status.endpoint);

You typically join a group in the onconnected callback:

serverConnection.onconnected = function() {
    this.join(group, 'join', username, password); 
}

After the server has replied to the join request, the onjoined callback will trigger. There, you update your user interface and request incoming streams:

serverConnection.onjoined = function(kind, group, perms, status, data, error, message) {
    switch(kind) {
    case 'join':
        this.request({'':['audio','video']});
        // then update the UI, possibly taking perms.present into account
        break;
    case 'change':
        // update the UI
        break;
    case 'redirect':
        this.close();
        document.location.href = message;
        break;
    case 'fail':
         if(error === 'need-username') {
            // the user attempted to login with a token that does not
            // specify a username.  Display a dialog requesting a username,
            // then join again
        } else {
            // display the friendly error message
        }
        break;
}

Sending and receiving chat messages

Once you have joined a group, you send chat messages with the chat method of the ServerConnection class. No permission is needed to do that.

serverConnection.chat(username, '', id, 'Hi!');

You receive chat messages in the onchat callback. The server may request that you clear your chat window, in that case the onclearchat callback will trigger.

Other messages

The usermessage method of the ServerConnection is similar to the chat method, but it sends an application-specific message. Just like chat messages, application-specific messages are not interpreted by the server; unlike chat messages, they are not kept in the chat history.

The useraction method is used to ask the server to act on a remote user (kick it, change its permissions, etc.); similarly, the groupaction class requests an action to be performed on the current group. Most actions require either the Op or the Record permission.

Accepting incoming video streams

When the server pushes a stream to the client, the ondownstream callback will trigger; you should set up the stream's callbacks here.

serverConnection.ondownstream = function(stream) {
    stream.onclose = ...;
    stream.onerror = ...;
    stream.ondowntrack = ...;
    stream.onstatus = ...;
}

The stream.label field is one of camera, screenshare or video.

After a new stream is created, ondowntrack will be called whenever a track is added.

The onstatus callback is invoked whenever the client library detects a change in the status of the stream; states connected and complete indicate a functioning stream; other states indicate that the stream is not working right now but might recover in the future.

The onclose callback is called when the stream is destroyed, either by the server or in response to a call to the close method. The optional parameter is true when the stream is being replaced by a new stream; in that case, the call to onclose will be followed with a call to onstream with the same localId value.

Pushing outgoing video streams

If you have the present permission, you may use the newUpStream method to push a stream to the server. Given a MediaStream called localStream (as obtained from getUserMedia or getDisplayMedia).

let stream = serverConnection.newUpStream();
stream.label = ...;
stream.onerror = ...;
stream.onstatus = ...;
localStream.getTracks().forEach(t => {
    c.pc.addTrack(t, c.stream);
});

The newUpStream method takes an optional parameter. If this is set to the localId property of an existing stream, then the existing stream will be closed and the server will be informed that the new stream replaces the existing stream.

Stream statistics

Some statistics about streams are made available by calling the setStatsInterval method and setting the onstats callback. These include the data rate for streams in the up direction, and the average audio energy (the square of the volume) for streams in the down direction.

Peer-to-peer file transfer

Galene's client allows users to transfer files during a meeting. The protocol is peer-to-peer: the clients exchange network parameters and cryptographic keys through the server (over messages of type usermessage), but all file transfer is performed directly between the peers.

An in-progress file transfer is represented by a JavaScript object of class TransferredFile. This object implements a finite state automaton whose current state is encoded as a string in the field state. It obeys the following state transitions:

(empty string) ⟶ inviting ⟶ connecting ⟶ connected ⟶ done ⟶ closed

(any state) ⟶ cancelled ⟶ closed

Downloading files

A client that wishes to participate in the file transfer protocol must set up the onfiletransfer callback of the ServerConnection object.

    serverConnection.onfiletransfer = function(transfer) {
        ...
    };

This callback will be called whenever a file transfer is initiated, either by the remote or by the local peer. The callback receives a single value of class TransferredFile. It should start by setting up the onevent callback, which is called whenever the state of the transfer changes and whenever data is received:

    transfer.onevent = func(state, data) {
        ...
    };

The direction of the file transfer is indicated by the value of the boolean this.up, which is false in the case of a donwload.

The callback may immediately reject the file transfer by either throwing an exception or by calling transfer.cancel and returning. If the file transfer is not immediately rejected, the callback should set up an onevent callback on the TransferredFile object:

    transfer.onevent = func(state, data) {
        ...
    };

It must then arrange for either transfer.receive or transfer.cancel to be called, for example from an onclick callback.

The onevent callback will then be repeatedly called, which can be used e.g. to present a progress bar to the user. Eventually, the onevent callback will be called with state equal to either cancelled or done; in the latter case, the transferred data is passed as a Blob in the data parameter of the callback.

Uploading files

A file upload is initiated by calling the sendFile method of the class ServerConnection.

serverConnection.sendFile(userid, file);

The userid parameter is the id of the remote peer. The file parameter is an object of kind File, typically obtained from an HTMLInputElement with type file.

The onfiletransfer callback is then called (with this.up set to true), and the transfer proceeds analogously to a file download, except that no data is passed to the onevent callback at the end of the transfer.

— Juliusz Chroboczek https://www.irif.fr/~jch/