Let’s build a Spotify GraphQL Server – Part 1

Update 1:

• Hint about blocked non-authenticated REST calls by Spotify. The demo and source code on github are already adapted, more details in a follow-up blog.
• improved code syntax-highlighting

GitHub built a GraphQL API server. You can write your own, too. This article shows how to write a GraphQL Server for Spotify:

1. How a simple Express Javascript server with the GraphQL endpoint can be set up
2. How to setup a graphql schema and what it is
3. Steps for implementing the proxy for retrieving / serving the data – spotify-graphql-server on Github

What is GraphQL?

The main motivation for developing GraphQL was the need to have efficient and flexible client-server communication

• GraphQL was built by facebook to have an advanced interface for specific communication for mobile clients requirements
• GraphQL is a query language
• GraphQL works as a central data provider (aka. single endpoint for all data)

See more details on graphql.org

As an example, let’s build a simple Spotify server, focussing on a mobile client with only minimum data needs.

Instead of fetching all needed data from these different endpoints by the client

https://api.spotify.com/v1/search?type=artist
https://api.spotify.com/v1/artists/{id}
https://api.spotify.com/v1/artists/{id}/albums
https://api.spotify.com/v1/albums/{id}/tracks

we will load these data per fetching from one GQL endpoint.

• Only one request: Each http request in mobile communication is expensive, because of the higher latency / ping times.
• Only the minimum data are transfered: This saves bandwith, because it avoids over-fetching all unneeded data, compared to the full REST response.

To achieve this, we will

• have a Javascript Express server, with Facebook’s reference implementation of graphql-js,
• add a GraphQL schema
• fetch the data from the Spotify API endpoint and aggregate them (asynchronously) on our server

1. Let’s start our project with a simple server setup

import express from 'express';
import expressGraphQL from 'express-graphql';
import schema from './data/schema';

const app = express ();
app.use('/graphql', expressGraphQL(req => ({
    schema,
    graphiql: true,
    pretty: true
})));

app.set('port', 4000);
let http = require('http');
let server = http.createServer(app);
server.listen(port);

This still needs a schema:

2. GraphQL Schema

“HTTP is commonly associated with REST, which uses “resources” as its core concept. In contrast, GraphQL’s conceptual model is an entity graph” (http://graphql.org/learn/serving-over-http )

A GraphQL schema consists of a list of type definitions.

For a minimalistic schema it needs a root node (aka Query type) which provides (indirect) access to any other data nodes in a graph.

Schema definition variant 1

In the following implementation, the query type has just one field hi which represents just the string hello world:

// after adding these imports:
import {
    GraphQLSchema,
    GraphQLString as StringType,
} from 'graphql';

// minimalistic schema

const schema = new GraphQLSchema({
    query: new GraphQLObjectType({
        name: 'Query',
        fields: {
            hi: {
                type: StringType,
                resolve: () => 'Hello world!'
            }
        }
    })
});

Let’s use the schema from above in our simple express server, start it with the command

babel-node server.js

and run a simple query with curl:

curl 'http://localhost:4000/graphql' \
     -H 'content-type: application/json' \
     -d '{"query":"{hi}"}'

If everything worked fine, we should get a JSON response where the query result can be found in the data property, any error information could be found in an optional error property of the response object.

{
  "data": {
    "hi": "Hello world!"
  }
}

Because graphiql was enabled on server start, you can simply point your browser to

http://localhost:4000/graphql?query={hi} and get the following page:

If we add some descriptions, GraphQL allows inspection. Let’s add a schema description and see how it works:

import {
    GraphQLString as StringType,
} from 'graphql';

const schema = new GraphQLSchema({
    query: new GraphQLObjectType({
        name: 'Query',

        description: 'The root of all queries."',

        fields: {
            hi: {
                type: StringType,

                description: 'Just returns "Hello world!"',

                resolve: () => 'Hello world!'
            }
        }
    })
});

We get code completion, and additional swagger like API documentation for free, which is always up-to-date!

We can fetch the built-in schema, as graphiql does it in the background:

The built in schema can be fetched per

curl 'http://localhost:4000/graphql' \
     -H 'content-type: application/json' \
  -d '{"query": "{__schema { types { name, fields { name, description, type {name} } }}}"}'

which gives us a long JSON response… hard to read (for humans).

So we should help us out of this mess by creating another representation using the printSchema module, which gives us a nice, readable output.

To create a more readable form, let’s create another representation using printSchema from graphql:

import { printSchema } from 'graphql';
import schema from './schema';

console.log(printSchema(schema));

This prints this format:

# The root of all queries."
type Query {
  # Just returns "Hello world!"
  hi: String
}

You might recognize that this is the same format as it is used when defining types in flowtype (type annotations for javascript)

Schema definition variant 2

This can even be used as a shorter, alternative approach to setup the schema:

import { buildSchema } from 'graphql';

const schema = buildSchema(`
#
# "The root of all queries:"
#
type Query {
  # Just returns "Hello world!"
  hi: String
}
`);

Then we just need to define all resolvers in the rootValue object, like the hi() function:

app.use('/graphql', expressGraphQL(req => ({
    schema,
    rootValue: {
      hi: () => 'Hello world!'
    },
    graphiql: true,
    pretty: true
})));

We could use arguments in the resolvers and could return a Promise. This is great, because it allows that all data fetching can run asynchronously! – But let’s postpone further discussion about timing aspects yet.

We will see this in our following example in the queryArtists resolve of Query later.

Let’s develop the real schema.

The most basic components of a GraphQL schema are object types, which just represent a kind of object you can fetch from your service, and what fields it has. from Schemas and Types

import { buildSchema } from 'graphql';

const schema = buildSchema(`
#
# Let's start simple.
# Here we only use a little information from Spotify API
# from e.g. https://api.spotify.com/v1/artists/3t5xRXzsuZmMDkQzgOX35S
# This should be extended on-demand (only, when needed)
#
type Artist {
  name: String
  image_url: String
  albums: [Album]
}
# could also be a single
type Album {
  name: String
  image_url: String
  tracks: [Track]
}
type Track {
  name: String
  preview_url: String
  artists: [Artists]
  track_number: Int
}

# The "root of all queries."

type Query {
  // artists which contain this given name
  queryArtists(byName: String = "Red Hot Chili Peppers"): [Artist]
}
`);

This graphql schema cheatsheet  by Hafiz Ismail gives great support.

Here we defined concrete Types, and also their relations, building the structure of our entity graph:

1. Uni-directional connections, e.g. albums of a type Artist: To specify this relation, we just define it as a field, of a specific type array of Albums. Any artist itself will be found when we use the query field/method with the query parameter: So, starting from the top-query, any node in our complete entity graph can be reached, e.g. Let’s fetch all tracks of any album of a specific artist by this query:

   {
   queryArtists(byName:"Red Hot Chili Peppers") { 
     albums {
       name
       tracks {
         name
         artists { name }
       }
     }
   }
   }
   

2. Even while the GraphQL can only provide tree-like data, the query can also fetch data from a cyclic graph. As you can see in the Track‘s artists field, above, or similar to fetching the followers of all followers of a Twitter user…

3. Resolvers – how to fill with data

Quick start with a mocking server

Because the schema in GraphQL provides a lot of information, it can be directly used to model sample data for running a mock server!

To demonstrate this, we use the mockServer from graphql-tools library and create dummy data dynamically:

import { mockServer, MockList } from 'graphql-tools';

let counter = 0;
const simpleMockServer = mockServer(schema, {
    String: () => 'loremipsum ' + (counter ++),
    Album: () => {
        return {
            name: () => { return 'Album One' }
        };
    }
    }
});

const result = myMockServer.query(`{
  queryArtists(artistNameQuery:"Marilyn Manson") { 
    name
    albums {
      name
      tracks {
        name
        artists { name }
      }
    }
  }
}`).then(result => console.log(JSON.stringify(result, '  ', 1))));

You can try it yourself with our source project per:

npm run simpletest

Result:

{
 "data": {
  "queryArtists": {
   "name": "loremipsum 1",
   "albums": [
    {
     "name": "Album One",
     "tracks": [
      {
       "name": "loremipsum 3",
       "artists": [
        {
         "name": "loremipsum 4"
        },
        {
         "name": "loremipsum 5"
        }, 
        "..." ] } ] } ] } } }

See apollo’s graphql-tools mocking for how to tweak it in more details.

Of course, we can use some more sophisticated mock data, but this was already quite usable when we would like to start any client-side development on top of our schema!

Retrieving and serving real data

To get real data, we need to implement these resolver functions, just like the hi function above. These functions will be used to retrieve any data from any data source. We can use arguments for access further query parameters.

In our case, let’s start with the query for an artist by name:

We just have the queryArtists resolver implementation:

//...
const app = express();
app.use('/graphql', expressGraphQL(req => ({
    schema,
    rootValue: {
        queryArtists: (queryArgs) => fetchArtistsByName(queryArgs)
    }
    // ...
})));

We could have add the resolvers into the schema definition as we did in the first variant above, but for less complex schemas like this one I prefer the second variant. It lets me split the logic for data fetching from the type definitions.

Any ‘field’ in rootValue corresponds to a ‘top query’ with the same name. Currently we only have the queryArtists.

Different kinds of resolvers:

• it can be any constant value/object: e.g. "Hello world."
• it may be a function: e.g. new Date()
• it may be a function returning a Promise which gets resolved asynchronously: e.g. fetch("from_url")
• it can be omitted, if the value can be derived from a property of parent object by same name automatically: In fact, any artist’s fields which was returned from fetchArtistsByName get returned directly.

This allows us to use and integrate all the powerful libraries out there without much effort! It’s easy to use mongoose, any github, twitter, or other client libs!

Here we have our own small implementation for fetching information per Spotify REST API.

In this query, we use the argument byName:

{
    rootValue: {
        queryArtists: ({ byName }) => {
            // using ES6 destructuring which is shorter and allows
            // default values
            return fetchArtistsByName(byName);
        }
    }
}

Just to get an impression, here we can see how to query the REST api of Spotify:

Important note: As Spotify requires to use authenticated requests by with OAuth Token (Bearer…) in the request header (background, details here ), this example does not work any longer out of the box. The project’s source code on github already contains the adaptions, more details about that in a follow-up post…

import fetch from 'node-fetch';

const fetchArtistsByName = (name) => {
   const headers = {
     "Accept": "application/json",
     // this won't work, just as an example
     // see updated code on github for a solution
     // this will be described a follow-up post
     "Authorization": "Bearer BQBg2HM1Q..."
   };
  return fetch('https://api.spotify.com/v1/search?q=${name}&type=artist')
    .then((response) => {
        return response.json();
    })
    .then((data) => {
        return data.artists.items || [];
    })
    .then((data) => {
        return data.map(artistRaw => toArtist(artistRaw));
    });
};

With the information in the raw JSON response we need to create the list of Artists per toArtist(). Reminder: any fields with same name as in schema will be used automatically!

const toArtist = (raw) => {
    return {
        // fills with raw data (by ES6 spread operator):
        ...raw,

        // This needs extra logic: defaults to an empty string, if there is no image
        // else: just takes URL of the first image
        image: raw.images[0] ? raw.images[0].url : '',

        // .. needs to fetch the artist's albums:
        albums: (args, object) => {
            // this is similar to fetchArtistsByName()
            // returns a Promise which gets resolved asynchronously !
            let artistId = raw.id;
            return fetchAlbumsOfArtist(raw.id); // has to be implemented, too ...
        }
    };
};

Summary:

We created our own Graphql server which loads basic information from the Spotify API server for us. Now, we can start fetching artists’ data in Graphiql: This should work with our local server per http://localhost:4000/graphql?query=%7B%0A%20%20queryArtis…. or per https://spotify-graphql-server.herokuapp.com/graphql?query=…. which gives you this result:

I will update the published version at spotify-graphql-server. It is based on the sources of spotify-graphql-server on Github , so you can play around with the latest version with latest features developed in this blog posts. Have fun!

In this article we already saw these main advantages of GraphQL:

• It allows us to specify exactly which data we need (no “over-fetching”)
• Extend the schema driven by the requirements of the client (think of Open-Closed principle)
• It defines a contract which always allows to verify the query against the schema definition (this works also at build time!)

In the next blog posts we will find out, how to

• build our own client, backed on free libraries from the awesome graphql eco system
• use standard express features for authentication for personalized infos (like play lists) …
• improve performance by caching on the server side using dataLoader
• adapt the schema for Relay support and use Relay on the client side,
  which can even be used in a React Native mobile client!
• we should check out mutations for even write access.