Reactive GraphQL

Update: You can start using GraphQL with Apollo today!

The core of Meteor has always been its advanced yet easy to use data system, which makes it very easy to efficiently publish reactive data. Looking around, it’s clear to us that many developers across the wider software community — for example, apps that use a combination of Angular and Spring, or React and Rails — have similar needs. We’re eager to bring that technology to a broader array of apps, and we’re also excited to upgrade Meteor with better control over reactivity, support for more data sources and clients, and enhanced productivity for developers.

We have listened to all of the feedback and ideas from the Meteor community’s experience with the current system, talked to production users of Meteor and other platforms, thought about our own experience building apps like Meteor Galaxy, and put together all we have learned in the last 4 years about building modern connected-client apps.

We are working on a set of modules that can be incrementally adopted in existing Meteor apps or in any JavaScript app that comprise a next-generation data loading platform that builds on the successes of Meteor’s DDP. Read on for more.

Common problems with data loading

Data loading is an integral part of all modern web and mobile apps. As users expect more from these apps, some common problems come into sharp focus. The issues below are not unique to Meteor, but gain prominence when building any reactive, connected-client JavaScript app.

Multiple data sources — SQL, REST, and MongoDB

Meteor currently only supports MongoDB. First-class support for other databases and backends has been one of the top most-requested features, and there are two main problems to be solved. The first is that your UI should be able to work independently of how the backend data is stored, which means you need a unified query language that can work to fetch data from many different sources. The second is that the number of data sources will always be larger than the number of maintained libraries, so it needs to be easy to write your own connector to any database or web API without needing a PhD in systems engineering.

Associations between objects

Most applications have to deal with combining associated objects. In Meteor’s DDP, you have to write complicated publications that publish all of the related data at once. In REST, you have to make multiple roundtrips to the server since each endpoint only returns one type of item, or you need to break out of the REST model and write custom endpoints. Wouldn’t it be great if your data system supported these associations as a first-class feature, along with other things that server-side frameworks take for granted, like pagination and aggregations?

Controlled reactivity

Many apps today have collaborative UI features that need a live view of data as it changes on the server. Four years ago, Meteor made the audacious choice to have live-updating as the default, and one of the hallmarks of a Meteor app is an engaging, collaborative experience. However, certain live queries have a performance tax and can cause UI complications if overused, and it’s hard to turn them off in Meteor when you don’t need them. In REST, achieving live updating can be a difficult task where different parts of your UI can get out of sync. Wouldn’t it be great if you could turn reactivity on and off when you need it, with hooks for optimizing the performance at scale?

Query composition from the UI

To keep your code easy to read, you want UI components to be able to define their own data requirements, but aggregate those into one efficient data fetch from the client. We love the way Relay has achieved this, and think it is the right mental model for maintainable UI code.

What we’re building

We talked to Meteor developers, UI engineers building with Angular and React, and even teams managing mobile apps with REST backends. Everyone has encountered some or all of the problems mentioned above in their applications. We think there is an opportunity to dramatically improve the landscape of application data management, both in the Meteor platform and for JavaScript apps in general, by building with GraphQL. Many people in the JavaScript and Meteor community are already excited about GraphQL, and we share that excitement based on a careful analysis of all of the options available.

Here’s what we are planning to build:

1. Simple and efficient bindings for GraphQL that make it easy to publish data from REST services, SQL, and MongoDB.
2. Reactive GraphQL queries that track changes to their underlying data, so clients can find out when the results have changed.
3. Easy client-side query aggregation and fetching so that you can compose and run reactive GraphQL queries on the client, regardless of your view technology — Blaze, React, or Angular.
4. Optimistic UI support based on lessons learned from 4 years of building optimistic systems within Meteor.
5. Low-level hooks and options for analyzing and optimizing your queries that make it straightforward to scale your app.

UPDATE: This is ready now, and it’s called Apollo.

Incremental and modular

Our goal is to build everything incrementally, so that every intermediate step is useful, and make the system as modular as possible so that the technology can be used in all kinds of tools, while still having a cohesive experience. We want to make something that is approachable and fast to develop with, while still having the tools to optimize your backend for scale.

Integration

One of our main goals is to make this new system easy to integrate with existing API technologies and view technologies. This means that while we are developing the new system, you should feel confident to keep using current methods and know that the migration will be easy.

1. If you have an existing Meteor app with subscriptions and methods over DDP, you will be able to use them side-by-side with a Meteor Data GraphQL endpoint. This can be a good option to add SQL to your app, or to use a mix of reactive and non-reactive queries. In the long run, we expect that the new system can be a complete and straightforward replacement for Meteor’s current pub/sub mechanism, with lots of the new features that Meteor users have been asking for.
2. If you have an existing Rails/PHP/Spring/Node app with REST endpoints, you can put a Meteor Data GraphQL server in front of it, so that you can use a GraphQL client to access the same endpoints but through a nicer, more performant API.
3. If you are using a client-side data management system such as Redux, it will be easy to integrate server data loaded using Meteor Data.

What this means for you today

Follow along and pitch in to the discussion. We envision a system that will be incrementally adoptable when and where you need it, so you don’t have to worry about rewriting any of your already working code. If the project reaches a stage that is useful to you, please give it a try and let us know how it goes.

We are going to be working with people that have existing complex production applications written with Meteor and other platforms to ensure that adopting the new system is as clear and smooth as possible, and doesn’t cause undue fatigue for productivity-conscious developers. We will also make sure the the low-boilerplate getting-started experience available with Meteor today remains that way, so you’ll be able to build your prototypes just as quickly as before.

Working together

We’re excited about all of the activity in the Meteor and JavaScript communities around GraphQL, and we’re excited to work together towards the best possible future. We’ve been inspired and informed by great efforts by people like Arunoda, and look forward to great discussions and collaborations with anyone who is interested.

Next steps

We’re excited to hear what you think of this idea, and what you have learned about application data loading in the last few years. We’re still working on the details of our design and implementation plan, and will post about it soon. In the meantime, please file issues on our Meteor Data GitHub repository or start discussions on the Meteor Forums. See you there!