Reacting to server events version 2

After several months in production, we have rolled out version 2 of our SSE feature.
Let's look at our lessons learned together.

Server Sent Events

Since the last blog, we have introduced version 2 of SSE within our apps. This helped us lower network data usage and also optimize the battery life for our users.

Network optimization

In version 1 our server would append change event records to a google firebase list. We would start a new list each day, and remove the old lists after a while.

Each addition would trigger firebase to send an update event to all subscribed clients.

When a client (browser) connect the first time, it would receive the whole list. Then it had to skip old events, using the timestamp in the event record.

This solution was fully functional, however it led to over 10GB of monthly network data usage. And, it turns out we did not really need any information except for the name of the object (table) timestamp in the record change event.

We decided to change the implementation from a list of events to a dictionary (map) of object-name: last-change-timestamp. This had the effect of dramatic network usage reduction. And it meant less work on the client, when old events had to be skipped.

Payload

Let's look at what is sent on the wire in version 2.

Here is the full payload sent by Firebase SSE on first connect.

event: put
{
   "path":"/",
   "data":{
      "activity":1718282112,
      "document":1718095416,
      "fi_appointment":1718781169,
      "fi_family":1718724505,
      "fi_family_history":1718707215,
      "fi_person":1718730128,
      "fi_person_history":1718728773,
      "proj_project":1710776262,
      "proj_task":1718282110,
      "proj_watcher":1710777390,
      "systemuser":1717570028,
      "usm_appointment_other_work":1718113622,
      "usm_appointment_person":1718720658,
      "usm_appointment_user":1718722632,
      "usm_appointment_work":1717658917,
      "usm_case":1718729029,
      "usm_case_issue":1718729029,
      "usm_case_support_user":1710940026,
      "usm_external_support_user":1712753777,
      "usm_family_type":1710929741,
      "usm_issue":1711530569,
      "usm_person_type":1712050711,
      "usm_project":1713424188,
      "usm_support_user":1718643585,
      "usm_support_user_person":1718172252,
      "usm_user_region":1717569935
   }
}

And here is the json payload when a record changes.

event: patch
data: {"path":"/","data":{"fi_person_history":1718783522}}

As you can see, the data is fairly minimal.

Server Implementation

Naturally since the wire format changed, we had to update the NodeJS server implementation. In fact, the code got a lot shorter and simpler, which is a win by itself.

Pay close attention to the fetch init object's method. We have to use PATCH, so that we only add or update properties and don't replace the whole object!

Note You can find the getFirebaseToken implementation in the previous blog.

export interface IExecuteRequest {
	name: string;
	object: any; // map of dbcolumn: value
	operation?: "create" | "update" | "delete";
}

export const sendPushNotifications = async (orgName: string, requests: IExecuteRequest[]) => {

	const groups: { [key: string]: number } = {};
	const unixTime = (new Date().valueOf() / 1000) | 0;
	for (const r of requests) {
		if (r.object) {
			groups[r.name] = unixTime;
		}
	}
	
	try {

		const token = await getFirebaseToken();
		const url = "https://<YOUR_FIREBASE_DB>.firebasedatabase.app/" + orgName + "/.json?auth=" + token;
		const init = {
			"body": JSON.stringify(groups),
			"method": "PATCH",
			"headers": {
				"Content-Type": "application/json",
			}
		}
		const resp = await fetch(url, init);
		const text = await resp.text();
	}
	catch (err) {
		console.log(err);
	}
}

Client (browser) implementation

Again, the code got shorter and more readable.

let instance: EventSource | undefined;

const handleSSEvent = (d: string, startDate: number) => {

	if (!d) return;

	try {
		const wrapper = JSON.parse(d) as { path: string, data: { [key: string]: number } };
		if (!wrapper.data)
			return;


		const changeObject = wrapper.data;
		for (const key of Object.keys(changeObject)) {
			const timestamp = changeObject[key];
			const objectName = key;

			if (+timestamp >= startDate) {
				window.dispatchEvent(new Event("objectChanged:" + objectName));
		}
	}
	catch (ex) {
		console.log("SSE handle put exception: " + ex + "\ndata: " + d);
	}
}

export const startPushListenService = async (orgName: string, start?: Date) => {

	console.log("SSE STARTING");
	try {
		const tokenResp = await fetch("/api/ssetoken"); 
		const token = await tokenResp.text();

		const startDate = start || new Date()
		const startTimeStamp = startDate.valueOf() / 1000;

		const url = "https://<YOUR_FIREBASE_DB>.firebasedatabase.app/" + orgName + "/.json?auth=" + token;
		const evtSource = new EventSource(url);
		evtSource.addEventListener('open', function (e) {
			//console.log("SSE open");
		}, false);
		
		evtSource.onmessage = (event: any) => {
			//console.log("SSE EVENT: " + event.data);
		};
		evtSource.onerror = (event: any) => {
			//console.log("ERROR SSE EVENT: " + event.data);
		}

		// called each time a record is modified, with the corresponding object name and change timestamp.
		evtSource.addEventListener("patch", function (e) {
			//console.log("SSE Patch UP - " + e.data);
			handleSSEvent(e.data, startTimeStamp);
		}, false);
		
		// called on startup with *all* objects (tables) and their last modified timestamps.
		evtSource.addEventListener("put", function (e) {
			//console.log("SSE Put UP ");
			handleSSEvent(e.data, startTimeStamp);
		}, false);

		if (instance)
			instance.close();
		instance = evtSource;
	}
	catch (e) {
		console.log("ERROR testSSE " + e);
	}
}

Inactivity optimization

After deploying the V1 of this feature, we started seeing surprising things in our logs. Devices that were supposed to be asleep, kept on querying our backend server. Turns out, that at-least OSX will keep SSE active during sleep (when on AC power for example). Another slightly less surprising issue was, that SSE was active even for a background (inactive) browser tab. This is a waste of network usage and battery life so we decided to optimize our SSE feature - only run while the tab is active. Thankfully, the modern web provides a very straightforward way to implement this change.

The document visibilityChanged event.

The concept of the feature is very straightforward. We shutdown server-sent-events listener while the documents inactive. And we restart once the docuemnt is active again (at the timestamp we’ve previously stopped listening).

This change will conserve the battery life of our users' devices. The devices can sleep without interruptions. And no unnecessary SSE handling code is running while a tab is inactive. Refreshing a list when nobody is looking is pointless and energy-wasting.

Below is the additional code. As you can see, it is short and easy.

let restartParams: { orgName: string, start: Date } | undefined;

const visibilityChanged = () => {
	if (document.visibilityState === "hidden") {
		if (instance) {
			instance.close();
			instance = undefined;
			/*
			* Record when we stopped listening for events.
			* We shall ignore events that happend before this timestamp once the documents is visible again (see below).
			*/
			restartParams.start = new Date();
		}
	} else if (document.visibilityState === "visible") {
		if (!instance && restartParams) {
			/*
			* Restart the service, ensure we skip very old events, but also replay events that happened while the doc was inactive.
			*/
			startPushListenService(restartParams.orgName, restartParams.start);
		}
	}
}

export const startPushListenService = async (orgName: string, start?: Date) => {

	try {
	
		// code cut for brevity ...

		const startDate = start || new Date()
	
		// code cut for brevity ...
		restartParams = { orgName, start: startDate };

		document.removeEventListener("visibilitychange", visibilityChanged);
		document.addEventListener("visibilitychange", visibilityChanged);

		// code cut for brevity ...

		if (instance)
			instance.close();
		instance = evtSource;
	}
	catch (e) {
		console.log("ERROR testSSE " + e);
	}
}

Wrap

Always up-to-date information is a great user experience. We still have to be mindful of the cost - additional network usage and processor time. In V2 we’ve optimized our always-fresh-data feature. Now we still keep the information uptodate, but only when the user cares about it - she is actively working with our app.

As the old saying goes with great power comes great responsibility. SSE can certainly give our apps great power, and we have to be careful how we wield it.

Happy hacking!