Finagle Essentials

@travisbrown

Introduction

What we’re doing
How we’ll do it
Getting set up

Goals

Learn to think in terms of Finagle’s core abstractions
Develop a deeper understanding of Try and Future
Learn how to write a Thrift IDL and generate bindings with Scrooge
Understand the challenges of working with blocking code in the context of Finagle

Approach

This is a hands-on lab!
Some slides, lots of code, some whiteboard
Follow along in the Scala REPL
Interrupt with questions at any time

Side note

This slide deck is built with CDK
Press t for a table of contents or h for more options
Visit the GitHub repository to view or edit the source

Getting started

What you need

Git
JDK (7+)

Cloning the project

git clone https://github.com/finagle/finagle-example-name-finder.git

Downloading the models (more about this soon)

cd finagle-example-name-finder
sh download-models.sh

Useful links

Build system

Note: we’ll use SBT today

This is mostly irrelevant
This isn’t a course about build tools
You could also use Pants

What’s a REPL?

REPL == "Read Eval Print Loop"
SBT runner is included here, and you can start it with ./sbt
From the SBT console, console will open a REPL
That’s (almost) all you need to know about SBT

What should it look like?

travis@sidmouth finagle-example-name-finder(master)$ ./sbt
[info] ...
> console
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import com.twitter.util.Try
import com.twitter.util.Try

Named-entity recognition

What is it?
What is OpenNLP?

Our example project

Start with a Java library that finds names in text
Wrap it in an idiomatic Scala API
Expose its functionality as a Finagle service

What is named-entity recognition?

Kinds of "names" commonly recognized

People
Places
Organizations
Monetary values

Example input

On account of the bequest of the late Ezekiah Hopkins, of Lebanon, Pennsylvania, U. S. A., there is now another vacancy open which entitles a member of the League to a salary of £4 a week for purely nominal services.

Example output

Pipeline

OpenNLP

Java Natural language processing library providing NER
Not as accurate out of the box as e.g. Stanford NLP, but…
Apache License, version 2.0
Easy to train new models for specific domains

Example usage

val sentDetector = new SentenceDetectorME(new SentenceModel(sdStream))
val tokenizer = new TokenizerME(new TokenizerModel(tokStream))
val finder = new NameFinderME(new TokenNameFinderModel(nfStream))

val sentences = sentDetector.sentDetect(document)
val tokenized = sentences map { s => tokenizer.tokenize(s) }
val nameSpans: Seq[String] = tokenized map { tokens =>
  Span.spansToStrings(finder.find(tokens), tokens)
}

finder.clearAdaptiveData()

Limitations of the API in this context

Lots of methods throw exceptions
Processing is synchronous
Not thread-safe

java.lang.IllegalArgumentException: The span [268..276) is outside
  the given text which has length 155!

Goal for our example project

Handle errors gracefully
Scale to take advantage of multiple processors
Scale to take advantage of multiple machines

Writing a Scala wrapper

Modeling the possibility of failure with types

Handling errors with exceptions

def parseAndIncrement(input: String): Int = input.toInt + 1

Modeling failure as a value

def parseAndIncrement(input: String): Try[Int] =
  Try { input.toInt } map { i => i + 1 }

Chaining computations that may fail

def safeDivide(n: Int, d: Int): Try[Int] = Try { n / d }

val good = for {
  n <- parseAndIncrement("5")
  d <- parseAndIncrement("1")
  result <- safeDivide(n, d)
} yield result

val bad1 = for {
  n <- parseAndIncrement("5")
  d <- parseAndIncrement("-1")
  result <- safeDivide(n, d)
} yield result

val bad2 = for {
  n <- parseAndIncrement("v")
  d <- parseAndIncrement("1")
  result <- safeDivide(n, d)
} yield result

Desugaring

val good = for {
  n <- parseAndIncrement("5")
  d <- parseAndIncrement("1")
  result <- safeDivide(n, d)
} yield result

val sugarFreeGood = parseAndIncrement("5").flatMap { n =>
  parseAndIncrement("1").flatMap { d =>
    safeDivide(n, d)
  }
}

Other methods

val tries = Seq("1", "2", "3").map(parseAndIncrement)

Try.collect(tries)

bad2.getOrElse(0)

bad2.rescue {
  case t: NumberFormatException => com.twitter.util.Return(0)
}

Relationship to Option and Either

Option: container of one or zero elements
Either: one of two types of things
Try: container of one element or an exception

Try in the Scala standard library

Semantically (almost) identical
Some of the names are different

Testing with ScalaTest

Using FunSuite, assert, and ===

Introduction to Finagle

Futures
Services
Servers
Clients
Filters

Futures

Like Try, but with an extra state

Not yet completed
Failed
Successfully completed (or "satisfied")

Futures, try, etc.

Future combinators

Like Try, can be combined using map, flatMap, handle, rescue, etc.

Also allows registration of callbacks:

onSuccess(f: A ⇒ Unit)
onFailure(ex: Throwable ⇒ Unit)

How futures are executed

When and where do they run?
How can we control that?
How are they different from the futures in the standard library?

Pop quiz

When do these return?

import com.twitter.util.Future

val f1 = Future { Thread.sleep(5000) }
val f2 = Future { 0 }.map { _ => Thread.sleep(5000) }
val f3 = Future.value(Thread.sleep(5000))
val f4 = for {
  a <- Future { 1 }
  b <- Future { Thread.sleep(5000); 2 }
} yield a + b

Extra credit

When do these return?

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

val f1 = Future { Thread.sleep(5000) }
val f2 = Future { 0 }.map { _ => Thread.sleep(5000) }
val f3 = Future.successful(Thread.sleep(5000))
val f4 = for {
  a <- Future { 1 }
  b <- Future { Thread.sleep(5000); 2 }
} yield a + b

From the quickstart

val service = new Service[HttpRequest, HttpResponse] {
  def apply(req: HttpRequest): Future[HttpResponse] =
    Future.value(new DefaultHttpResponse(
      req.getProtocolVersion, HttpResponseStatus.OK))
}

We need to be careful with I/O, since Future.value blocks a Finagle thread

Rule of thumb (courtesy of Moses Nakamura)

Never await result of another Finagle request
90% blocking, 10% busy: put it on another thread
10% blocking, 90% busy: put it on another thread
0% blocking, 100% busy, very uneven workload: another thread
0% blocking, 100% busy, even workload: probably okay

Future pools

import com.twitter.util.FuturePool

val pool = FuturePool.unboundedPool

val f1 = pool { Thread.sleep(5000); 0 }
val f2 = pool { 0 }.flatMap { i => pool { expensiveOp(i) }}

Sequencing computations

for {
  foo <- getFoo()
  bar <- getBar(foo)
  baz <- getBaz(foo) // Don't do this!
} yield (bar, baz)

What’s wrong here?

One solution

for {
  foo <- getFoo()
  bar = getBar(foo)
  baz = getBaz(foo)
  barValue <- bar
  bazValue <- baz
} yield (barValue, bazValue)

A better solution

for {
  foo <- getFoo()
  pair <- getBar(foo).join(getBaz(foo))
} yield pair

"Applicative" sequencing

More applicative sequencing

Future.collect(futures: Seq[Future[Int]])

Services

A service is a function

class Service[-Req, +Rep] extends (Req => Future[Rep])

Try models failure as a value
Future models both failure and delay as a value

What services aren’t

The service API doesn’t know anything about the network

import com.twitter.finagle.Service
import com.twitter.util.Future

val parserService = new Service[String, Int] {
  def apply(request: String) = Future(request.toInt)
}

Servers

Servers make services available on the network over a protocol

import com.twitter.finagle.Http

val myHttpService: Service[HttpRequest, HttpResponse] = ???

val server = Http.serve(":8080", myHttpService)

Http is a Server[HttpRequest, HttpResponse]
server is a ListeningServer

Clients

The term "client" is overloaded

The Client interface creates "clients" (in the second sense) for a specific protocol
A "materialized client" is a ServiceFactory
In some cases the Services created by a ServiceFactory are called "clients"
Instances of Scrooge’s ThriftService are often called "clients"

Example: HTTP client

A client in our third sense

val client: Service[HttpRequest, HttpResponse] =
  Http.newService("www.google.com:80")

Filters

Filters have a complicated-looking type:

class Filter[-ReqIn, +RepOut, +ReqOut, -RepIn]
  extends (ReqIn, Service[ReqOut, RepIn]) => Future[RepOut]

Filters are actually relatively simple: they’re just service transformers

Timeout filter

TimeoutFilter is an example of a SimpleFilter (doesn’t change types)

import com.twitter.conversions.time._
import com.twitter.finagle.util.DefaultTimer
import com.twitter.finagle.service.TimeoutFilter

val myTimeoutFilter =
  new TimeoutFilter[String, Int](1.second, DefaultTimer.twitter)

Using a timeout filter

import com.twitter.util.FuturePool

val slowParserService = new Service[String, Int] {
  def apply(request: String) = FuturePool.unboundedPool {
    Thread.sleep(5000); request.toInt
  }
}

val myService = myTimeoutFilter andThen slowParserService

Protocols

Finagle is designed to make it possible to define many components in a protocol-agnostic fashion

We’ll be building servers and clients that speak the Thrift protocol today
The Finagle Quickstart gives an HTTP example
Other supported protocols include Redis, Protobuf, MySQL, SMTP, ZooKeeper, etc.
See finagle-serial for example with Mux as session-layer protocol
See finagle-smtp for example of custom protocol

Thrift and Scrooge

The Thrift interface description language allows us to define data types and service interfaces
Bindings for specific languages are created using code generation tools
We’ll be using Twitter’s Scrooge via an SBT plugin

Where’s the code?

Generated automatically when we compile with sbt compile
Lives in target/scala-2.10/src_managed/main

Implementing the Scrooge interfaces

We need to define a method implementation for every function in our service

Putting it all together

A bad solution
A better solution

A bad solution

What’s wrong with NaiveNameRecognizerService?

(Hint: there’s more than one thing)

A better solution

Use a future pool to keep Finagle threads free when we’re doing IO
Use resource pools to control access to our non-thread-safe objects

twitter-server

twitter-server: an alternative App

Flags for configuration
Logging and metrics
Admin HTTP interface
Lifecycle management endpoints for e.g. Mesos’s job manager

Using twitter-server

./sbt run

Then visit the admin page, metrics, etc. on port 9990