Enterprise Microservices with Python

Ken Yeatman, Principal Engineer, DigitalGlobe | Radiant
Jun 29 2017

We love Python and the speed of development it enables. But we also love interoperability.

At HumanGeo, we spend a great deal of time working with the latest technologies often in combination with easy-to-change, scripting languages like Python or Perl. These technologies enable us to deliver value to our customers while maintaining flexibility to rapidly adjust in an evolving environment.

One way we build solutions for our customers is with a microservice—a modular piece of code that can be deployed independently of other components to perform a function. We need our microservices to interoperate with the Java and Scala services built using the Finagle and Finatra stack from Twitter.

When I encountered an enterprise microservice environment through one of our customers, I decided to see if I could create a Python-based microservice that would meet our requirements and support the rapid creation of new services. In this post, I write about a microservice implementation in Python, and introduce the packages required to achieve interoperability and suitable performance and scale.

About microservices

A microservice is exposed via a lightweight interface (REST, Thrift or similar) and coupled together with other microservices to form an application. (Anyone remember web services?) To make them broadly usable in an enterprise, microservices usually end up with additional baggage in the areas of discoverability, security, monitoring (DevOps) and usability. In our specific case, already deployed microservices, built in Java, using Twitter’s Finatra and Finagle frameworks, provide a functional example of what Python needs to do.

Writing “Hello, World”

One of the great things about Python is the large number of already-built modules and frameworks ready for use. A microservice doesn’t need the bulk of an ORM framework, so Flask, Bottle or another light framework are great choices.

app/views.py

Wrapping it as a microservice that can scale and perform

To keep performance up, I selected asyncio-based pyGrowler based strongly on Flask) and augmented it with uvloop. (For those unfamiliar with uvloop, it replaces the asyncio event loop and can potentially double your performance compared with other approaches.) I ended up with the following code to work with:

run.py

app/_init_.py

What’s going on

It’s a pretty basic web application that is returning a couple of static views. In our environment, these views are pre-built help files containing the method descriptions of what our application is supporting (in this case a simple ping.) In this setup, the run.py file is responsible for the actual running of the application and management of the event loop, the __init__.py file for setting up the various pieces, and views.py for managing the responses of the service. The above code gives us a functioning HTTP server using Python asyncio with uvloop and a placeholder for our self-describing microservice (a requirement of our environment.)

Making it secure

Since running anything useful under HTTP is not a great idea, the next step is to add in SSL. In the client environment, two-way SSL is a requirement. We can take advantage of some of the built-in Python libraries and extend our startup by creating an SSL context. This context can then be passed directly into pyGrowler. Presto! SSL-enabled REST microservice.

Up next

Unfortunately, we are still a little short on the pieces our microservice needs to match the functionality of our target environment’s Finagle services. In my next post I will talk about adding in Zookeeper registration, exposing Thrift endpoints, and making everything behave in the server environment. Stay tuned…