Reacting to server events
Browser requests and the server responds.
This has been the norm for decades.
And still applies to modern web-apps.
Motivation
But what if our users want to collaborate and be aware of each other’s work?
Or what if we do some longer-duration calculation on the server and we want to tell users it was completed.
In general, what if we need the server to send data to the client without a client's request?
Good news is that, yes it is possible to do.
Even better news is you have multiple options for how to do it.
Options
- Web push notification
- WebSockets
- Server-sent events (SSE)
First, let's look at each in detail and then we will look at how to implement server-sent events.
Push notifications.
Might sound new, but they are widely supported even on mobile browsers.
The massive benefit is that your webapp needs not to be running for the notification to arrive.
However the user might not accept push notifications, or they can get lost.
Thus, push notifications are mostly a slick addition along the traditional email notifications.
Best suitable for scenarios when you would send an email to the user.
Just remember, it is not guaranteed that the push message will arrive.
See here for more details.
WebSockets
WebSockets are a general purpose bidirectional communication channel.
Very flexible, but very low level. Client and server send and receive messages.
What each message means and when to send what message is your responsibility to figure out.
This great flexibility is best suited for apps where collaboration with users is the center piece.
A great example is a multiplayer game, a chat app.
Or a shared whiteboard app, where multiple users can draw at the same time.
They are well supported in most browsers and http servers. However the protocol might get blocked by some firewalls.
A WebSocket is opened by the webapp in javascript.
Server-sent events
Conceptually, they are the missing part of the web communication to make it bi-directional.
As the name implies, they allow the server to send messages without a prior request from the client (webapp).
Again a connection has to be opened by the client in javascript.
SSE can be introduced along the classic request-response model.
That means we can continue using normal javascript fetch and start adding reactivity gradually.
Without disturbing the existing code.
In contrast with WebSockets, where we replace or invent our communication channel from scratch.
Another positive is that we can decide in javascript how we handle the incoming server sent message.
Whether we show a message to the user or just do some background processing.
The nice thing about SSE is the simplicity.
Let's first recap. In a standard http communication, a connection is opened by the client.
Who sends a request and then reads all data that the server sends in response.
In SSE the client opens the connection and sends a request - exactly as before.
But then the server will send only the http headers back. Both client and server leave the connection open.
When the server has a message for the client, he writers it into the connection.
The client-browser sees the new data and passes it to a provided javascript function to handle it.
Server-sent events implementation
*** We have an updated version. You can read more here: ***
Version 2
Motivation
Imagine a webapp that shows a page with for a footbal match.
Along the details of the match, we also want to see the highlights of the event (goals, cards, substitutions etc.).
Implementation
We will implement the webapp and backend server in Javascript.
But, for added scalability we shall use Google firebase realtime database for SSE.
This way our backend server is not burdened with active connections waiting for changes.
And, since we can get a realtime database for free, it allows us to quickly deploy and test without additional cost.
Client
To implement SSE on the client we can use the raw DOM api or a library.
Let's look at the raw API as that will be good enough for this example.
In the code below we will register for changes to an 'orgName' or organization.
For our demo app, an organization can be a country.
Imagine an app that has subdomains for countries like uk.latestnews.com, sk.latestnews.com etc.
For each country, we have a list of objects - the individual sport types, football, basketball.
We will start the service and monitor all changes within a country for the current day.
(We also make sure to restart the service at midnight.)
Once an event is received, we check whether a particular object - match changed (goal?!). Or a we got an aggregate event for multiple changes.
Finally we dispatch the event using the standard DOM window.dispatchEvent function.
interface IWrappedChangeNotification {
path: string;
data: IChangeNotification;
}
interface IChangeNotification {
timestamp: string; // Date().toISOString
objectName: string; // football, basketball, ...
tooManyChanges?: boolean; // massive write
ids?: string[]; // keys of modified objects - particular sport events/matches
}
const startSseService = async (orgName: string) => {
try {
const tokenResp = await fetch("/api/ssetoken");
const token = await tokenResp.text();
const startDate = new Date();
const day = startDate.getUTCDate();
const restartAt = (startDate.getUTCHours() * 60 + startDate.getUTCMinutes()) * 60 + startDate.getUTCSeconds() + 1;
window.setTimeout(() => startPushListenService(orgName), restartAt * 1000);
const url = "https://your-database.firebasedatabase.app/a" + day + "/" + orgName + "/changes.json?auth=" + token;
const evtSource = new EventSource(url);
evtSource.addEventListener("put", function (e) {
if (!e.data)
return;
const dispachChange = (change: IChangeNotification) => {
const dt = new Date(change.timestamp);
if (dt < startDate) {
//console.log("SSE skipping old event");
return;
}
window.dispatchEvent(new Event("objectChanged:" + change.objectName, change));
}
try {
const wrapper = JSON.parse(e.data) as IWrappedChangeNotification;
if (!wrapper.data)
return;
const seen: { [key: string]: IChangeNotification } = {};
if (wrapper.path === "/") { // aggregate event
const changes = Object.values(wrapper.data) as IChangeNotification[];
for (const change of changes) {
const objName = change.objectName;
if (!seen[objName])
seen[objName] = change;
else {
if (change.ids)
seen[objName].ids = (seen[objName].ids || []).concat(change.ids);
seen[objName].tooManyChanges = !!seen[objName].tooManyChanges || !!change.tooManyChanges;
}
}
for (const change of Object.values(seen)) {
dispachChange(change);
}
} else {
dispachChange(wrapper.data);
}
}
catch (ex) {
console.log("SSE handle put exception: " + ex + "\ndata: " + e.data);
}
}, false);
if (instance)
instance.close();
instance = evtSource;
}
catch (e) {
console.log("SSE startup error: " + e);
}
}
We will display sporting match info in two ways.
A list of matches and a detail of a match.
In the list we will simply re-fetch all matches when we get a change notification.
In the detail, we can check the id of the match to see whether we need to fetch news. We leave this part as an exercise for the reader:)
const MatchListComponent = (props: {objectName: string}) => {
const {orgName, objectName} = props;
const [records, setRecords] = useState([] as any[]);
const loadRecords = async () => {
const resp = await fetch("/api/matches/" + objectName);
const recs = await resp.json();
setRecords(recs);
}
useEffect(()=>{
loadRecords();
}, [objectName]);
useEffect(()=>{
const changeHandler = (e: any) => {
loadRecords();
}
const eventName = "objectChanged:" + objectName;
window.addEventListener(eventName, changeHandler);
return () => window.removeEventListener(eventName, changeHandler);
}, [objectName])
return <div className="objectGrid">
{records.map(r=>{
return <div key={r.id}>{r.name}</div>
})}
</div>
}
Now we shall look at the server side.
Implemented with NodeJS and Express.
We will call this method from our server-side code, whenever a match is created, updated or deleted.
interface IExecuteRequest {
name: string; // object name
operation: "create"|"update"|"delete";
object: { id: string };
}
export const sendPushNotifications = async (orgName: string, requests: IExecuteRequest[]) => {
// If we have changes to multiple objects, let separate them by object name.
const objectGroup: { [key: string]: string[] } = {};
for (const r of requests) {
if (r.object && r.object.id) {
if (!objectGroup[r.name])
objectGroup[r.name] = [];
objectGroup[r.name].push(r.object.id);
}
}
const timestamp = new Date().toISOString();
for (const kv of Object.entries(objectGroup)) {
const objectName = kv[0];
const ids = kv[1];
try {
const payload: IChangeNotification = {
timestamp,
objectName,
}
if (ids.length > 100)
payload.tooManyChanges = true;
else
payload.ids = ids;
const token = await getFirebaseToken(true);
const day = new Date().getUTCDate();
const url = "https://your-database.firebasedatabase.app/a" + day + "/" + orgName + "/changes.json?auth=" + token;
const init = {
"body": JSON.stringify(payload),
"method": "POST",
"headers": {
"Content-Type": "application/json",
}
}
const resp = await fetch(url, init);
const text = await resp.text();
//console.log("SSE sent:" + text);
}
catch (err) {
console.log(err);
}
}
}
Finally the code to get the access token for Firebase.
It is a little complicated because it can be called multiple times, while we are still waiting for the token.
// Singleton token, refreshed as needed.
const FirebaseToken: { expires?: Date, token: string, p?: Promise<string> } = { token: "" };
export const getFirebaseToken = (): Promise<string> => {
if (!FirebaseToken.token || !FirebaseToken.expires || FirebaseToken.expires < new Date()) {
if (FirebaseToken.p) {
return FirebaseToken.p;
}
FirebaseToken.p = new Promise<string>(async (res, rej) => {
const apiKey = "YOUR_FIREBASE_WEB_KEY"; // Firebase Project Settings
const url = "https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=" + apiKey;
const payload = {
email: "YOUR_FIREBASE_USER",
password: "YOUR_FIREBASE_PASSWORD",
returnSecureToken: true,
};
const init = {
"body": JSON.stringify(payload),
"method": "POST",
"headers": {
"Content-Type": "application/json",
}
}
try {
const resp = await fetch(url, init);
const j = await resp.json();
FirebaseToken.token = j.idToken;
FirebaseToken.expires = new Date(new Date().valueOf() + j.expiresIn * 900) // ask sooner than exacly as many seconds as reported
FirebaseToken.p = undefined;
res(FirebaseToken.token);
}
catch (e) {
rej(e);
}
});
return FirebaseToken.p;
}
return new Promise<string>((res) => res(FirebaseToken.token));
}
app.get("/api/ssetoken", mustAuth, async (req, res) => {
try {
const token = await getFirebaseToken();
res.send(token);
}
catch (e) {
res.status(500).send("ERROR: " + e);
}
})
Change is good, don't overreact
Traditional web pages and webapps are fresh only for a little while after the initial load.
Afterwards it is our job to make sure the user is not using outdated information.
These days collaboration is becoming the differentiating factor.
It can very well be the USP for your app.
I hope the above will help you on your way to build a fresh app.
Happy hacking!