How real-time data works on Meteor — Know your pub/subs

Let’s start with how publications and subscriptions work on Meteor, which is the “under the hood” of Meteor’s real-time data model.

It’s easier to understand when seeing the flow of new data, excluding client simulations:

You do a method call and change an object in a collection.Mongo oplog will propagate that change to observers that tails it (publications create observers on oplog).Publications observers detect oplog changes that affect its observed data, and it spawns a data re-fetch on mongo database.Meteor merge box will check the diffs and propagate it to the client, as added/changed/removed.Meanwhile, your method results might come after or before. It’s usually before, but if your oplog is fast and you have more computation after changing the DB, it might come after.Flow for real-time updates on Meteor full-stack architecture

Ok!

Now we have an overview of how a publication works and the server resources involved in getting real-time data from the server/database.

In most cases, you can also use Meteor Methods simulation, which runs the same code you run in the server, in the client, but targeting the minimongo. In this way, you don’t need to wait for the server response, and the client gets updated instantly.

But why is this so important to understand? If you want to have fast applications using real-time data, you have to know how it works and structure your application to take the best from every aspect. Let’s talk about the ideal architecture for publications, then.

How to structure your publications

The ideal architecture is:

Small publications only with reactive data. (Remember merge-box? Yes, it will compute every diff on change docs for every user. Publish data with care!)With data that is scoped. How scoped, you might ask, scoped as something you can restrict when invalidating.

Let’s grab an example:

A chat app. It might incline you to do a publication containing all messages from all chat groups. But, that would cause tremendous overhead on the server.

The ideal scenario is to do publications based on groups, like:subscribe(“MessagesForUserFromGroup”, groupId)

— more performant, will cause fewer cursor invalidations

Compared with

subscribe(“AllMessagesForUser”)

The goal of publications is to avoid cursor invalidations. It’s a normal course of publications, caused by changes in the underlying data layer, as we saw, but if you restrict what you publish or do it smartly, you can reduce it.

Also, the best thing is to think in your architecture so that you can reuse the same cursors, which is done when subscribing with the same parameters.

These are some high-level thoughts I have on publications! I hope it helps.

How real-time data works on Meteor — Know your pub/subs was originally published in Meteor Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.