Write an API with Django Rest Framework

The API built using DRF needs:

  • Router – connect requests to ViewSet
  • ViewSet – What am I getting from the database and displaying?
  • Serializer – How am I turning the data into a JSON object
from rest_framework import APITestCase

class TestMyModelViewset(APITestCase):
  def setUp(self):
     self.url = reverse('mymodel-list')
     self.instances = [MyModel() for i in range(3)]
     for item in self.instances:
         item.save()

  def test_list_view(self):
    response = self.client.get(self.url, format='json')
    self.assertEqual(response.status_code, 200)
    self.assertEqual(len(response.data), 3)

  def test_can_create(self):
    data = ('name': 'test name', 'category': 1)
    response = self.client.post(self.url, data, format='json')
    self.assertEqual(response.status_code, 201)

  def test_detail_view(self):
    sample_id = MyModel.objects.first().id
    detail_url = self.url + '/' + str(sample_id)
    response = self.client.get(detail_url, format='json')
    self.assertEqual(response.status_code, 200)

 

 

JavaScript @DjangoCon

Originally you had two tools:  transpile JS to turn it into ES5, then browserify it to one bundle.

Webpack is both of those tools plus more (CSS, images, assets, etc) for apps!

Rollup is the tool for libraries!

JS Starting stack:

  • Babel for transpilation
  • Webpack for bundling
  • ESLint for linting (or tsc)
  • yarn or npm for package management

Library: a codebase that you call from your code (Requests, HTTP)

Framework: a codebase that calls your code (Django)

Starting again:

create-react-app

  • Yarn compatible, Webpack & Babel, ESLint, and Jest
  • JSX and React
  • Redux must be installed, redux-logger, redux-react

Django views

RetrieveUpdateDestroyAPIView will do all the CRUD things

class CoinDetailView(RetrieveUpdateDestroyAPIView):
  permission_classes = (IsAuthenticated,)
  serializer_class = CoinSerializer
  queryset = Coin.objects.all()

That’s quick!

http://www.django-rest-framework.org/api-guide/generic-views/#retrieveupdatedestroyapiview

views.py

from .models import Coin
from .serializer import CoinSerializer

class CoinViewSet(mixins.createModelMixin, mixins.ListModelMixin, mixins.UpdateModeMixin, mixins.RetrieveModelMixin, mixins.DestroyModelMixin, viewsets.GenericViewSet):
  #stuff

 

router.py

from django.conf.urls import url, include
from django.contrib import admin

from home.views import CoinViewSet
from rest_framework.routers import DefaultRouter

router = DefaultRouter()
router.register('coins', CoinViewSet)

urlpatterns = [
  url(r'^api/', include(router.urls))
]

Just need to add the router.urls call

 

GraphQL in the Wild…

GraphQL is an interesting technology with a lot of good points, but some drawbacks.

Pros:

  • Self explorable
  • easy documentation
  • intuitive
  • simplifies client side logic

Cons:

  • Graphene is the only library and still young.
  • v1.0 released in Sept2016 and not very updated.
  • Docs are not very good
  • Known bugs with resolver – the order graphene does it by default is wrong
  • Source code is complicated with lots of meta-programming, so hard to update
  • Authorization
  • Denial of Service
  • Performance!

Performance issues with REST

Showing a lot of related data requires lots of requests

Serialization takes a lot of time, especially with REST resources that contain a lot of information that isn’t used on most of the pages

Dashboards Challenge

GraphQL vs REST! 5 s vs 10s result even after reducing the data sent by the REST.

GraphiQL app from Github

{
  viewer {
    login
    repositories (first:10) {
      edges {
        node {
          name
        }
      }
   }
}

Get only the data you want, strong typing, nested fields

With Django?

Graphene!

pip install graphene_django
INSTALLED_APPS += ['graphene_django']
urlpatterns += [url(r'^graphql', GraphQLView.as_view)]

Define nodes:

class TaskNode(DjangoObjectType):
  class Meta:
    model = Task

Whenever you have a field you can have a resolver for it that defines how to get the field.

Define queries:

class Query(ObjectType):
  class Meta:
    model = Task

Define the Relay:

{
  goals (first: 5, after: "cursor") {
    name
    progress
  }
}

Pagination

class GoalNode(DjangoObjectType):
  progress = graphene.Float(description='The average task progress')
  pk = graphene.Int()
  tasks = graphene.List(TaskNode)

  @graphene.resolve_only_args
  def resolve_pk(self):
    return self.pk

  @graphene.resolve_only_args
  def resolve_tasks(self):
    retrun self.key_results.filter(closed=False)

  class Meta:
    model = Objective
    filter_fields = ['name',]
    exclude_fields = ['key_results',]

class OwnerNode(DjangoObjectType):
    full_name = graphene.String()
 

 

GraphQL will display the description in the GraphiQL browser that’s useful!

Use cases that make it better than REST:

Complex views (dashboards, summaries, stats)

Complex Writes

Builders, autosaving, auto-updating.  Checks the types!  Define mutations!

Makes the change and then re-queries and returns the response in one call instead of many!

The backend knows what the mutation is and does it and the frontend just says what something should change to.

Make sure your mutations are @atomic!

Add it as a mutation on your schema.

Authorization

Performing authorization on each resolver is a pain in the ass, so you have to extend Graphene to authenticate on the connection.

Extend DjangoFilterConnectionField

connection_resolver: add user authentication

resolve_connection: get auth class from node and apply auth

Throttling/DOS

  1. Whitelist for allowed queries – all of the queries in the backend in python, and the frontend called the query by name.  (def blah, return graph query)
  2. Maximum limit
  3. Maximum query cost – first: 50 repositories, first 10 issues = 550 total nodes.
  4. Rate limiting based on query cost.  Based on the umber of database connections you’re getting.

Performance

Built to work, not to be performant.  Doesn’t use select_related or prefetch related. Needs to reduce the count() calls.

Data Loader (facebook’s answer to performance issues) – return a promise that analyses the data requested and reduces the requests made

goals (first: 0) {
   totalCount
}

Do the count in the graph instead of in the front end.

Need to write queries in a smart way!

Resources

  • graphql.org
  • Zero to GraphQL (video)
  • Intro to GraphQL (blog)

 

Django User Scenarios

#1 Proxy Model

Create an alias to an existing model.

Good: Custom model manager, custom model methods

Limitations: Can’t add additional attributes, can’t change the database

Author(User):
  class Meta:
    proxy = True

  def __str__(self):
    return self.first_name

Author is just like the user model, but has some methods that aren’t on the built in User class or that override the defaults.

#2 Using a one to one relationship

Create a new model that has a one to one relationship with the User Model.

Good: Custom fields

Limitations: Another model in the database (migration management), requires additional steps in the code to handle the model relations

class Profile(models.Model):
  user = models.OneToOneField(User)
  # add other custom fields desired
  location = ...
  bio = ...

Accessing:

user.first_name
user.profile.location

Need to add:

  • Handling User saves & updates & deletes
  • Handling Profile creation on User creation
  • Include in Django Admin

Most of these are included in the Django Documentation.

https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#extending-the-existing-user-model

Note: Create at the beginning of the project.

#3: Custom User Model

Good: Flexibility

Limitations: User Manager, Admin Form for the Custom User Model

class MyUser(AbstractBaseUser):
  email = ...
  fav_type = ...

  USERNAME_FIELD = 'email'

settings.py

AUTH_USER_MODEL = 'myapp.MyUser'

https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#substituting-a-custom-user-model

NOTE: Create custom user model at the beginning of your project

A full example of a custom User

https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#a-full-example

These are notes from #DjangoCon2017.  Here’s a link that goes through much more in depth for each of these options:

https://simpleisbetterthancomplex.com/tutorial/2016/07/22/how-to-extend-django-user-model.html

Event Driven Architecture with Lambda

The advantage of using lambda is there’s no queue system (celery) that causes no blocking.  You use events to spawn lambdas for each event that comes in.  Configure/use this with the zappa project.

To resize a thumbnail on a user uploading a picture, you run a function based on a save to s3.

events:
  - function: users.util.process_avatar
    event_source:
     arn: arn:aws:s3:::your_event_name

Don’t get stuck in an infinite loop!  Use two buckets (processed/raw), or set path of new object so that it doesn’t trigger your event.

If your function takes a long time to run, then you don’t want to use the event driven architecture because it’ll time out.  Use zappa.async import task.  API endpoint calls the task wrapped function, which then starts a new async server that runs the longer code and returns.

 

Django on Lambda

Use a project called zappa that makes it easy!

Gotcha #1 : Security!  ALLOWED_HOSTS

Automatically generated or subdomain added to ALLOWED_HOSTS

Gotcha #2: Static Files

Use django-storages and add ‘storages’ to the installed_apps, configure it, and then collect static, and zappa update

Gotcha #3: Database

  • Can you use a queue or something else instead?

nodb!

  • Can you use S3-database?

zappa-bittorrent-tracker using S3-database

  • No?

Use AWS RDS (expensive but easy) or EC2 (cheap but annoying).

Need a VPC to allow the lambdas but prevent random internet traffic. Two private subnets (for redundancy), Allow TCP on 5432 (postgres).

add vpc to django settings

set the host/port in the DATABASES setting in Django

Install zappa-django-utils to interact with the django database inside the VP

Gotcha #4: Encryption!

Use ACMI or LetsEncrypt  with zappa certify

 

Bringing Functional Programming into an Imperative World

This is notes from the Bringing Functional Programming into an Imperative World talk from DjangoCon 2017 – Derik Pell @gignosko on Github with demonstration repository at https://github.com/gignosko/DjangoCon_2017

Functional Programming is expressive, efficient, easier to work between concurrent/parallel programming (using more and more cores rather than faster processors), and increases safety in code if done well.

You can fake an immutable data structure using deepcopy… but it takes some extra time.

Recursion takes the place of loops in many functional languages

You can mimic functional programming with list comprehensions and lambdas

filter

[x for x in list_l if x % 2 == 0]

map

[(lambda x: x*x)(x) for x in list_1]

FP can be less verbose, more efficient, and more intuitive

BUT! can be slower, recursion can blow the stack up, functional code looks weird

Supercharging the Django Admin

Liam Andrew (mailbackwards) gave a talk at #DjangoCon 2017 about ‘Supercharging the Django Admin’.  Here is some of the information and a GitHub repo with a project demonstrating the information from the talk.

Ways to change the admin classes

  • Add list_display to your Admin class to add more columns to the header field.

Define a method and then add that to the list_display to have arbitrary information, but remember to add prefetch_related to models you load in that method.

  • Allow people to edit things from the admin list using `list_editable` but since this adds a button and a text field, it’s often overwhelming unless used for a specific purpose.

 

  • Make dynamic actions using a method that returns a select box depending on what the user is attempting to do.

 

Useful Packages

  • django-admin-row-actions : add a button/action at the end of each row for that individual item
  • django-object-actions : Put buttons in the changelist without editing the template.  Adds a changelist_actions to the admin models.
    return redirect(‘admin:newshound_breed_changelist’)
  • django-nested-admin: Allows multiple layers in one view (breed group, breed, or dog)  with sorting!
  • django-inline-actions: another way to add action buttons to each row

 

Tips

Beware of over-querying without prefetch!

Repository with concrete examples

https://github.com/mailbackwards/newshound