The web world moves quickly; every developer involved with this ecosystem is painfully aware of this fact. The state-of-the-art stack you start building your startup with today is considered ancient technology within the next 3 months. Even so, there are a few timeless (at least at Javascript-speed) gems produced in this world. GraphQL is a technology that has been around for about 2 years now and it integrates well with the React/Relay stack to provide a more complete ecosystem for a web developer to work in. While Facebook and others have done a fine job discussing the merits of GraphQL and why you would use it (read: I'll let them convince you it's a good idea), we're going to discuss how to harness its power in a very reproducible way.
Difficulties with GraphQL
GraphQL is a great piece of technology. However, as a developer you're left with few guidelines to get moving quickly. In and of itself, GraphQL is a very non-opinionated interchange format. More specifically, it describes a graph-based structure for accessing and modifying data. While data access may seem like a straightforward task when previewing GraphQL, mutations are vague and very under-spec'ed out of box. While in theory (and practice once you've wrapped your head around it) this is actually a good thing for maximum flexibility, it leaves many developers scratching their heads wondering what to do.
GraphQL gains much of its power from this broad flexibility. At the same time, this flexibility creates a lot of cognitive overhead on developers. To date, most-- if not all-- GraphQL API's I have seen are hand-written. This is a cumbersome task. As we move to client-side apps, the backend servers are becoming more like secure datastores which enforce business constraints on data. This was likely always the case in traditional web development as well, but the backend was often merged with UI and UI generation code; as a result, this distinction was blurred. The only other logic performed on these servers is now often private computations which should be concealed from the clients' code. Moreover, with hand-written endpoints, it's difficult to guarantee both API consistency and complete functionality from one endpoint to the next.
Ultimately, this creates an API ecosystem where each endpoint is different. Namely, understanding how to operate one endpoint within my own GraphQL application doesn't necessarily guarantee I can properly operate others (much less other GraphQL apps in general). This would then require copious amounts of documentation on each endpoint for any developer to get up and running effectively. This is more reminiscent of my hardware days-- scanning through datasheets of similar components-- than it is of developing highly generic and reusable software components.
Solving these GraphQL Difficulties
As luck would have it, we already have a solution to eliminating the boilerplate for hand-written endpoints. In 2015, we built Elide. Elide enables developers to model their data using JPA, communicate with arbitrary datastores, secure it using meaningful expressions, and write custom code where necessary. It's been a large-scale, multi-year effort, but it has proven very effective on our own products. In any case, this is a solution designed to solve all the problems that fallout from hand-written endpoints: API consistency, uniform functionality, proper security, developer boredom, etc. The only problem: it didn't support GraphQL (and is only now in the 4.0 beta).
Initially, when we went out to build Elide, rather than reinventing the wheel we sought a standardized web API solution. GraphQL hadn't yet gained any traction (and iirc, it wasn't even fully released). As a result, we found JSON API and decided to build Elide around supporting this technology. It was opinionated and generating the API was straightforward. While JSON API has a lot of strengths, we also found some issues with the opinionated stances in which it took (more on that in a later post). Now you may be thinking to yourself, "Wait a minute, why is this important? I thought this post was on GraphQL." Well, it turns out that working so intimately with JSON API, we used-- what we believe to be-- some of its best ideas to inform ourselves on how to generate uniform, consistent, and automatable GraphQL API's.
Now that we have discovered a solution for minimizing endpoint code, there was a new looming question. How do we generate a consistent GraphQL API? That is, for any endpoint, all you need to know is the data model. From then on, you will have a standard set of tools available to you. Of course you could extend any particular model with special logic as needed, but generally speaking, there would exist a fully supported, common toolset for all models.
Designing a GraphQL API
Our motivation for building out GraphQL support in Elide was to more easily adopt its client-side ecosystem. It will not replace JSON API, but instead live alongside it for users to choose which solution is best for their project. However, this imposes an additional set of constraints; subtle implementation details such as pagination must be compatible with the existing tooling (i.e. Apollo or Relay). But one problem at a time: while GraphQL query-schemes appear to be well worked out, we first need to solve the problem for having a consistent means of object mutation.
Consistent GraphQL API Mutation
Our first goal was to find a consistent way of specifying GraphQL mutations; basically, any time you wish to insert or update data. After several ideas, we came back to an approach inspired by JSON API and REST (though it is certainly not REST). In summary, we turn each JPA relationship and Elide rootable types into objects which take GraphQL arguments. The arguments are op, ids, and data. Without going into all the details (the current spec can show examples), this allows us to support the same operations in a decidable way across all exposed entities in our GraphQL API. A brief description of each parameter is below:
- op. This parameter describes the operation to be performed. When unspecified, this defaults to a FETCH operation, but it can also take on values such as UPSERT, REMOVE, REPLACE, and DELETE.
- ids. When provided, this list of id's is used to filter the collection on which the operation is being performed.
- data. The data argument is used for UPSERT'ing and REPLACE'ing. It specifies the new input data from the user.
With these three arguments, a user can perform arbitrary data operations on their models.
An Apollo/Relay-Ready GraphQL API
Automating an API that is Apollo- and Relay-compatible adds a few more layers of complexity. In short, a model in our originally proposed scheme would look like:
|
1 2 3 4 5 6 7 8 |
book(op: FETCH, ids: ["1"]) { title authors { name } } |
The example above will fetch a book object from the system with id and id of 1. It will return its title and all of the names of its associated authors. Pretty straightforward, right?
Well, when accounting for important concepts like pagination and so forth, this will not work with Relay out of box. As a result, we adopted Relay's Cursor Connections for maximum compatibility. While the scheme is still what we have proposed, there are now 2 additional layers of indirection for each model containing metadata (namely, edges and node objects). These layers are used for additional metadata. See below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
book(op: FETCH, ids: ["1"]) { edges { node { title authors { edges { node { name } } } } } } |
As you can see, the layers of indirection make the format a little bit uglier, but the same overall concepts apply.
Conclusion
GraphQL has many great ideas and an incredible amount of flexibility. However, it's difficult to avoid writing many hand-written endpoints and maintaining consistency across all of them. However, Elide was invented to solve one of these problems and has recently implemented a consistent method for generating GraphQL API's. If you're looking for a quick, out-of-box solution, I recommend considering Elide for your next project (to get started, see the example standalone project). In any case, when you build your next GraphQL API, be sure to think through these problems. If you don't adopt our scheme directly, at least be aware of the problems it solves. Good luck out there!
