From 8697555125c57ae64a0c9b78514b4aac4fd523de Mon Sep 17 00:00:00 2001 From: Mitja Felicijan Date: Tue, 27 Jun 2023 14:50:20 +0200 Subject: Massive formatting and added figcaption --- .../2020-03-22-simple-sse-based-pubsub-server.md | 132 +++++++++++---------- 1 file changed, 68 insertions(+), 64 deletions(-) (limited to 'content/posts/2020-03-22-simple-sse-based-pubsub-server.md') diff --git a/content/posts/2020-03-22-simple-sse-based-pubsub-server.md b/content/posts/2020-03-22-simple-sse-based-pubsub-server.md index 77c42a9..60745d0 100644 --- a/content/posts/2020-03-22-simple-sse-based-pubsub-server.md +++ b/content/posts/2020-03-22-simple-sse-based-pubsub-server.md @@ -7,35 +7,35 @@ draft: false ## Before we continue ... -Publisher Subscriber model is nothing new and there are many amazing solutions -out there, so writing a new one would be a waste of time if other solutions +Publisher Subscriber model is nothing new and there are many amazing solutions +out there, so writing a new one would be a waste of time if other solutions wouldn't have quite complex install procedures and weren't so hard to maintain. -But to be fair, comparing this simple server with something like -[Kafka](https://kafka.apache.org/) or [RabbitMQ](https://www.rabbitmq.com/) is +But to be fair, comparing this simple server with something like +[Kafka](https://kafka.apache.org/) or [RabbitMQ](https://www.rabbitmq.com/) is laughable at the least. Those solutions are enterprise grade and have many -mechanisms there to ensure messages aren't lost and much more. Regardless of -these drawbacks, this method has been tested on a large website and worked -until now without any problems. So now, that we got that cleared up, let's -continue. - -***Wiki definition:** Publish/subscribe messaging, or pub/sub messaging, is a -form of asynchronous service-to-service communication used in serverless and -microservices architectures. In a pub/sub model, any message published to a +mechanisms there to ensure messages aren't lost and much more. Regardless of +these drawbacks, this method has been tested on a large website and worked until +now without any problems. So now, that we got that cleared up, let's continue. + +***Wiki definition:** Publish/subscribe messaging, or pub/sub messaging, is a +form of asynchronous service-to-service communication used in serverless and +microservices architectures. In a pub/sub model, any message published to a topic is immediately received by all the subscribers to the topic.* ## General goals - provide a simple server that relays messages to all the connected clients, - messages can be posted on specific topics, -- messages get sent via [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) +- messages get sent via [Server-Sent + Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) to all the subscribers. ## How exactly does the pub/sub model work? -The easiest way to explain this is with diagram bellow. Basic function is -simple. We have subscribers that receive messages, and we have publishers that -create and post messages. Similar model is also well know pattern that works -on a premise of consumers and producers, and they take similar roles. +The easiest way to explain this is with diagram bellow. Basic function is +simple. We have subscribers that receive messages, and we have publishers that +create and post messages. Similar model is also well know pattern that works on +a premise of consumers and producers, and they take similar roles. ![How PubSub works](/assets/simple-pubsub-server/pubsub-overview.png) @@ -45,43 +45,47 @@ on a premise of consumers and producers, and they take similar roles. - consumer is receiving messages from subscribed topic, - servers is also known as Broker, - broker does not store messages or tracks success, -- broker uses [FIFO](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)) - method for delivering messages, -- if consumer wants to receive messages from a topic, producer and consumer topics - must match, +- broker uses + [FIFO](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)) method + for delivering messages, +- if consumer wants to receive messages from a topic, producer and consumer + topics must match, - consumer can subscribe to multiple topics, - producer can publish to multiple topics, - each message has a messageId. **Known drawbacks:** -- messages will not be stored in a persistent queue or unreceived messages like - [DeadLetterQueue](https://en.wikipedia.org/wiki/Dead_letter_queue) so old +- messages will not be stored in a persistent queue or unreceived messages like + [DeadLetterQueue](https://en.wikipedia.org/wiki/Dead_letter_queue) so old messages could be lost on server restart, -- [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) - opens a long-running connection between the client and the server so make - sure if your setup is load balanced that the load balancer in this case can - have long opened connection, +- [Server-Sent + Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) + opens a long-running connection between the client and the server so make sure + if your setup is load balanced that the load balancer in this case can have + long opened connection, - no system moderation due to the dynamic nature of creating queues. ## Server-Sent Events -Read more about it on [official specification page](https://html.spec.whatwg.org/multipage/server-sent-events.html). +Read more about it on [official specification +page](https://html.spec.whatwg.org/multipage/server-sent-events.html). ### Current browser support ![Browser support](/assets/simple-pubsub-server/caniuse.png) -Check [https://caniuse.com/#feat=eventsource](https://caniuse.com/#feat=eventsource) +Check +[https://caniuse.com/#feat=eventsource](https://caniuse.com/#feat=eventsource) for latest information about browser support. ### Known issues - Firefox 52 and below do not support EventSource in web/shared workers -- In Firefox prior to version 36 server-sent events do not reconnect +- In Firefox prior to version 36 server-sent events do not reconnect automatically in case of a connection interrupt (bug) -- Reportedly, CORS in EventSource is currently supported in Firefox 10+, - Opera 12+, Chrome 26+, Safari 7.0+. +- Reportedly, CORS in EventSource is currently supported in Firefox 10+, Opera + 12+, Chrome 26+, Safari 7.0+. - Antivirus software may block the event streaming data chunks. Source: [https://caniuse.com/#feat=eventsource](https://caniuse.com/#feat=eventsource) @@ -104,7 +108,7 @@ data: this is line two ``` -And you can specify your own event types (the above messages will all trigger +And you can specify your own event types (the above messages will all trigger the message event): ```bash @@ -116,7 +120,7 @@ data: 103.34 ### Server requirements -The important thing is how you send headers and which headers are sent by the +The important thing is how you send headers and which headers are sent by the server that triggers browser to threat response as a EventStream. Headers responsible for this are: @@ -129,23 +133,23 @@ Connection: keep-alive ### Debugging with Google Chrome -Google Chrome provides build-in debugging and exploration tool for -[Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) +Google Chrome provides build-in debugging and exploration tool for [Server-Sent +Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) which is quite nice and available from Developer Tools under Network tab. -> You can debug only client side events that get received and not the server -> ones. For debugging server events add `console.log` to `server.js` code and +> You can debug only client side events that get received and not the server +> ones. For debugging server events add `console.log` to `server.js` code and > print out events. ![Google Chrome Developer Tools EventStream](/assets/simple-pubsub-server/chrome-debugging.png) ## Server implementation -For the sake of this example we will use [Node.js](https://nodejs.org/en/) -with [Express](https://expressjs.com) as our router since this is the easiest -way to get started and we will use already written SSE library for node -[sse-pubsub](https://www.npmjs.com/package/sse-pubsub) so we don't reinvent -the wheel. +For the sake of this example we will use [Node.js](https://nodejs.org/en/) with +[Express](https://expressjs.com) as our router since this is the easiest way to +get started and we will use already written SSE library for node +[sse-pubsub](https://www.npmjs.com/package/sse-pubsub) so we don't reinvent the +wheel. ```bash npm init --yes @@ -252,16 +256,16 @@ app.listen(port, () => { ### Our custom message format -Each message posted on a server must be in a specific format that out server -accepts. Having structure like this allows us to have multiple separated type -of events on each topic. +Each message posted on a server must be in a specific format that out server +accepts. Having structure like this allows us to have multiple separated type of +events on each topic. -With this we can separate streams and only receive events that belong to the +With this we can separate streams and only receive events that belong to the topic. -One example would be, that we have index page and we want to receive messages -about new upvotes or new subscribers but we don't want to follow events for -other pages. This reduces clutter and overall network. And structure is much +One example would be, that we have index page and we want to receive messages +about new upvotes or new subscribers but we don't want to follow events for +other pages. This reduces clutter and overall network. And structure is much nicer and maintanable. ```json @@ -278,15 +282,15 @@ nicer and maintanable. -You can download [the code](../simple-pubsub-server/sse-pubsub-server.zip) -and follow along. +You can download [the code](../simple-pubsub-server/sse-pubsub-server.zip) and +follow along. ### Publisher -As talked about above publisher is the one that send messages to the -broker/server. Message inside the payload can be whatever you want (string, -object, array). I would however personally avoid send large chunks of data -like blobs and such. +As talked about above publisher is the one that send messages to the +broker/server. Message inside the payload can be whatever you want (string, +object, array). I would however personally avoid send large chunks of data like +blobs and such. ```html @@ -359,19 +363,19 @@ like blobs and such. ### Subscriber -Subscriber is responsible for receiving new messages that come from server via -publisher. The code bellow is very rudimentary but works and follows the +Subscriber is responsible for receiving new messages that come from server via +publisher. The code bellow is very rudimentary but works and follows the implementation guidelines for EventSource. -You can use either Developer Tools Console to see incoming messages or you can +You can use either Developer Tools Console to see incoming messages or you can defer to Debugging with Google Chrome section above to see all EventStream messages. -> Don't be alarmed if the subscriber gets disconnected from the server every -> so often. The code we have here resets connection every 15s but it -> automatically get reconnected and fetches all messages up to last received -> message id. This setting can be adjusted in `server.js` file; search for -> the `maxStreamDuration` variable. +> Don't be alarmed if the subscriber gets disconnected from the server every so +> often. The code we have here resets connection every 15s but it automatically +> get reconnected and fetches all messages up to last received message id. This +> setting can be adjusted in `server.js` file; search for the +> `maxStreamDuration` variable. ```html -- cgit v1.2.3