A brief history of APIs at dailymotion

  1. A Public Graph API that allows our API clients to access and manage videos, collections, uploads…
  2. A new internal GraphQL API for our brand-new Dailymotion experience.

Scaling the API design for performance

Fig1. The explosion of consumption models
Legacy Dailymotion mobile apps
GET /video/x68r2tt{
"id": "x68r2tt",
"title": "Stranger Things - Brings 80's Fashion Back",
"owner": "x1x1syp",
"description" : "Stranger Things is causing a fashion storm as many of their fans are borrowing clothes from their parents and dressing like a blast from the past.",
...
...
}
  • Over-fetching issue: requesting the entire ressource to get just a small piece of data.
  • Chattiness issue: multiple API requests are needed to build mobile views, which could be hindering the global performance of our mobile implementations.

Introducing the Graph API

  • Each object exposes multiple fields of different types (scalar, objects…)
  • An object field can be readeable, writeable and/or used as a filter.
  • Objects can be connected to each other.
Graph API Pattern
GET /video/x68r2tt?fields=title,description,owner.username{
"title": "Stranger Things - Brings 80's Fashion Back",
"description" : "is causing a fashion storm as many of their fans are borrowing clothes from their parents and dressing like a blast from the past.",
"owner.username": "FYINews",
}
limit chattiness
Field deprecation
Track API usage with Kibana
curl -I https://api.dailymotion.com/videos
HTTP/1.1 200 OK
Cache-Control: public, max-age=900
....
X-Cache: HIT
Vary: X-DM-EC-Geo-Country, X-DM-SSL,Accept-Encoding
curl -I "https://api.dailymotion.com/videos?live&fields=onair,audience"
HTTP/1.1 200 OK
Cache-Control: public, max-age=10
....
X-Cache: MISS
Vary: X-DM-EC-Geo-Country, X-DM-SSL,Accept-Encoding

Migrating to GraphQL

  • We had exactly 6 months to build a new API.
  • The new API had to be geo-distributed at its core and from the start.
  • It needed to be easy to consume for our front developers.
  • It needed to be easy to code and deploy for our core api developers.
  • It represented an opportunity to build from scratch a service architecture.
  • We already expose our data as a Graph in our Graph HTTP API, and GraphQL already declares everything as a Graph.
  • GraphQL query operation is very interesting because it allows the WYWIWYG (What you want is what you get), thus our API clients will have more flexibility to request their data requirements.
  • The resolver feature brings a good design approach to use the underlying services.
  • Some tools, SDKs are already built around this technology : GraphiQL, Apollo SDKs, and “Check this list
GraphQL powered experience

Lessons learned

  • Creating a non open standard API has a cost! Simply because you have to build from scratch every tool needed and associated documentation to on-board your developers.
  • Treat your API as a product, and invest in making your API easy to use and understand.
  • Maintaining API SDKs is really hard if your API team is small
  • Prioritise performance and scalability for your use case when picking an API design paradigm.
  • Use RUM (Real User Monitoring) for your API to get insights into your API performance, and it will also help you answer key questions : how does API really perform from a specific country or in a specific device…
  • If you have a monolith stack and you have a performance issue check first what can be done, without a full migration.
  • Give freedom to your core team to break things, each time someone breaks our production they put money into a so-called “BeerBox” that we empty each 4 months to celebrate.
  • Don’t blindly migrate to GraphQL, if you don’t know the consequences. This techno may fix some of your issues but brings others issues that you have to deal with (Cacheability, API Lifecycle, Rate-Limit, Security…).
  • GraphQL is not a silver bullet to fix your API design. You really have to introduce an API Design guide from the beginning.
  • Dealing with mutations is a pain, because of the unpredictability of types, for instance if you want to update a Video Node ? what is the mutation without checking documentation ? (VideoUpdate, UpdateVideo, PatchVideo…).
  • Don’t see GraphQL spec with REST eyes, it’s a completely a different approach with its benefits and its limits.
  • Don’t try to compare GraphQL with REST and just forget all REST vs GraphQL debates, unless you like to compare apples and oranges!

Scaling the API team

The First API Squad
  • Every core developer can expose optimised features in the API because he has the expertise to do it.
  • The API Squad has provided training sessions for new API Badge members, so they can collaborate as fast as possible in the API.
  • Our API Squad only reviews the API design and the API contract.
  • We provided cool “API Badge” branded T-shirts for API Badge members ;)

Lessons learned

  • If you have a dedicated API team think twice and define clearly its mission statement.
  • Very good communication skills are needed for your API Core Developers, consider them as API Evangelists first! before coders.
  • Enabling ownership of the API across teams was painful! but it was worth it at the end.
  • If you scale fast you will encounter some chaos and complexity to handle in your organisation and if you ask your self “Wow things are different” or “Why things are not done ?”, think about re-designing your organisation!

--

--

--

Helping organisations on how to get from an API vision and strategy to a functioning and effective API platform including people, technology and governance.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How to schedule tasks on Linux with CRON

Getting Started with Okteto and Rust

Our Team is Growing!

Active Database and Trigger In DBMS

How I saved my relationship with Scrum?

Maven Multimodule Project: A Detailed View

Grin: Hardfork retrospective, RFC process & sub-teams update, Security Process Update, Niffler…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Samir AMZANI

Samir AMZANI

Helping organisations on how to get from an API vision and strategy to a functioning and effective API platform including people, technology and governance.

More from Medium

Using Bearer Token Authentication

Developer Showcase #2: Matias Lugli, Developer Manager at Tavano

Matias Lugli

How to use Authorizer with React Native Expo?