How to Deploy Prophet by Facebook on AWS Lambda

A quick look at Prophet and how it can help you

As Prophet is implemented as R and Python procedure, the focus for AWS Lambda is on the Python 3.7 runtime.

.__version for pandas, numpy and fbprophet imports

Hello World

Depending on how you came to this story you might be asking, “What is Prophet?”

Let’s quote its creators:

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

After testing the model locally, I got great results that made me decide to bring it to a production level. I tried to avoid ground-level work to set up an AWS EC2 instance. Therefore, I decided to go with AWS Lambda. In hindsight that sounded slightly easier than it was.

Here’s how it’s done.

What’s the Catch?

The main challenge to be achieved in one line explained is import fbprophet . AWS Lambda out of the box obviously doesn’t come with that package preinstalled. Therefore we must do it. I found the official documentation helpful, but not sufficient to get it done.

The reasons are:

• The package must be installed on a replicate of the live AWS Lambda environment. Here I got stuck at first as I was “just” following the official documentation with my macOS. However, the deployment failed due to differences to the Amazon Linux environment. While one option certainly is to set up an EC2 instance online, I found an easier solution with docker-lambda. It’s a docker image to replicate the AWS Lambda environment.
• Prophet needs PyStan. Since version 2.19 Stan requires a compiler which supports C++14. Long story short, we need gcc 4.9.3 or higher. As of now Amazon Linux 1 based Lambdas use gcc 4.8.5. Another time I got stuck. I figured the easiest solution was to pip install pystan==2.18 instead. Another possible solution is mentioned here.
• As of now, deployment packages on AWS Lambda can be max. 250 MB (unzipped, including layers). So we can’t just upload the package — we have to delete certain folders to bring the packages to that size. Credit to Alexandr who did most of the heavy lifting in his story here.

Let’s Get it Done

Since you’ve read this far, you might be experiencing similar problems. I tried to create a plug and play solution so you can get back to the fun part quicker than I could.

1. Clone or download my repository. In it, you will find a Dockerfile that will resolve all the problems I mentioned above.
2. Open the folder of the repository that you just downloaded. In here you can also find the filelambda_function.py. No need to edit it right now, but be aware of it. That’s the file where you can change your lambda_handler .
3. Now, open your terminal and cd into the repository. Run the following command:

docker build -t fbprophet . && \
docker run --rm -v $PWD:/export \
fbprophet cp upload-to-s3.zip /export

This can take a few minutes. Once done, a new file upload-to-s3.zip should appear in the repository folder. That’s it. With that .zip-file you will be able to import fbprophet on AWS Lambda. As the file is larger than 50MB, you must upload via an S3 bucket. In my first tests, I chose 1024 MB Memory and 10 seconds Timeout for my Lambda function. Here’s an example set up:

aws lambda create-function --function-name fbprophet \
--runtime python3.7 --handler lambda_function.lambda_handler \
--timeout 10 --memory-size 1024 \
--role arn:aws:iam::[INSERT]:role/[INSERT]

If that part is not familiar to you I found these tutorials helpful:

Tutorial: Using AWS Lambda with Amazon S3

There is also sample code for Python for that specific tutorial.

Just testing? Run this command and get a ready to upload upload-to-s3.zip file for AWS Lambda:

docker run --rm \
-v $PWD:/export \
marcmetz/prophet:1.0 \
cp upload-to-s3.zip /export

Obviously, it’s only an example lambda_function.py in it. But you can already create a “Hello World” test event to see if the import is working. (See screenshot on top and section Invoke the Lambda Function.)