Introducing Formplug v1, a form forwarding service for AWS Lambda
Daniel Ireson

Introducing Formplug v1, a form forwarding service for AWS Lambda

Sat Jan 20 2018

It’s estimated that approximately 269 billion emails are sent in a single day. Over 10 million were sent as you read that previous sentence. Even in 2018, over 40 years after its creation, email is still one of the most reliable and versatile means of communication on the internet.

Throughout this blog post I’ll be providing a technical walkthrough of the latest release of Formplug, an open-source form forwarding service that I’ve been working on. It makes it incredibly simple to receive form submissions by email. Typically you’d use Formplug in an environment where you don’t have the capability to execute sever-side code, like Github Pages for example.

There is an online demo if you wish to see it in action, or for getting started instructions you can view the project on GitHub.

Screenshot of the demo website

How it works

Upon deploying the Formplug service to AWS you will receive an API Gateway endpoint URL. This should set as the form action for any forms that you want to receive by email. Behaviour can then be customised using hidden form inputs prefixed by an underscore. For example, to set the recipient you would use an input named _to.

Here's an example of what a minimal form might look like:

<form action="https://apigatewayurl.com" method="post">
    <input type="hidden" name="_to" value="johndoe@example.com">
    <input type="text" name="message">
    <input type="submit" value="send">
</form>

Among other features, there’s support for encrypting email addresses, spam prevention and URL redirection. See the readme for the full range of configuration options.

Architecture

A Lambda in the context of AWS can be thought of as a function that gets invoked based on a predefined event.

Formplug is made up of two Lambdas:

  1. A receive Lambda — for parsing form submissions.
  2. A send Lambda — for building and sending emails.

In the case of the receive Lambda, events are generated from form submissions to our API Gateway endpoint (a public URL which is generated on the first deploy). In the case of the send Lambda the event is generated from an API call from the receive Lambda.

Formplug architecture

Configuration

Both of these Lambdas are defined in the serverless.yml configuration file, which describes the required AWS infrastructure. The infrastructure will be automatically instantiated on deployment using CloudFormation.

functions:
  receive:
    handler: src/receive/handler.handle
    events:
      - http:
          path: /
          method: post
          request:
            parameters:
              querystrings:
                format: true
  send:
    handler: src/send/handler.handle

For each Lambda we’re defining an exported JavaScript function as the handler. This will be invoked each time the Lambda’s event is triggered. The handler itself is just an arrow function which takes three arguments, the first of which is event which in our receive handler holds the form data itself.

module.exports.handle = (event, context, callback) => {

}

Whenever a form is posted to our endpoint, this function is invoked. In order to return a response we need to use the provided callback argument. This is an error-first callback so we should pass null as the first argument for a successful response. The second argument should be an object containing a status code and response body. The callback defaults to the application/json content type.

callback(null, {statusCode: 200, body: 'Email sent'})

Validating the form submission

At the top of the Formplug receive handler, a new request instance is created and the event is passed into the constructor. The Request class is responsible for validation. Defined in the constructor is everything that needs to be extracted from event. This is primarily just the recipients for the email and the form inputs to be forwarded on.

class Request {
  constructor (event) {
    this.singleEmailFields = ['_to']
    this.delimeteredEmailFields = ['_cc', '_bcc', '_replyTo']
    this.recipients = {
      to: '',
      cc: [],
      bcc: [],
      replyTo: []
    }

    this.responseFormat = 'html'
    this.redirectUrl = null

    this.pathParameters = event.pathParameters || {}
    this.queryStringParameters = event.queryStringParameters || {}
    this.userParameters = querystring.parse(event.body)
  }
}

We don’t do any real work in the constructor. Also notice how we use the Node.js querystring module to set userParamters. This takes the request body containing the form data encoded as application/x-www-form-urlencoded and turns it into a JavaScript object.

Validate using promises

There’s a validate method on Request that’s responsible for parsing the event. It takes advantage of the ability to sequentially chain promises. Each success handler in the promise sequence is used to validate a different aspect of the form submission. Rejected promises bubble up to their top-level parent, so we can define a single catch method in the handler that’s shared amongst all of the validation methods.

validate () {
  return Promise.resolve()
    .then(() => this._validateResponseFormat())
    .then(() => this._validateNoHoneyPot())
    .then(() => this._validateSingleEmails())
    .then(() => this._validateDelimiteredEmails())
    .then(() => this._validateToRecipient())
    .then(() => this._validateRedirect())
}

If you’re wondering what one of these specific validation methods looks like, let’s look at _validateSingleEmails which is responsible for setting the _to recipient. It checks whether any singleEmailFields have been provided in userParameters, resolving the promise if the validation was successful and rejecting the promise if there were errors.

_validateSingleEmails () {
  return new Promise((resolve, reject) => {
    this.singleEmailFields
      .filter((field) => field in this.userParameters)
      .forEach((field) => {
        let input = this.userParameters[field]
        if (!this._parseEmail(input, field)) {
          let msg = `Invalid email in '${field}' field`
          let err = new HttpError().unprocessableEntity(msg)
          return reject(err)
        }
      })

    return resolve()
  })
}

HTTP errors are rejected

Instead of rejecting a JavaScript error directly in the promise, an error is generated using HttpError. This class has been created to provide a friendly API for generating common HTTP response errors. It contains a public method for each supported error response. Shown below is an example of the class with just the 422 unprocessable entity response for brevity.

class HttpError {
  unprocessableEntity (message) {
    return this._buildError(422, message)
  }

  _buildError (statusCode, message) {
    const error = new Error(message)
    error.statusCode = statusCode
    return error
  }
}

You might be thinking that it’s not worth structuring error handling like this as requires more upfront work. The benefits become more apparent when you consider how clean the receive handler can become. We can reject a HttpError anywhere in the promise chain and have it caught in the top-level catch - at this point an appropriate response can then be built and returned.

Generating responses

To generate a response it’s not quite as simple as providing a status code and message to callback. Both JSON and HTML content types are supported, each require different headers to be set and have a different body format. Redirect responses through the _redirect form input are also supported, which similarly has different requirements. Response has been created that can build the different objects that should be passed to callback.

class Response {
  constructor (statusCode, message) {
    this.statusCode = statusCode
    this.message = message
  }

  buildJson () {
    return {
      statusCode: this.statusCode,
      headers: {
        'Access-Control-Allow-Origin': '*',
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        statusCode: this.statusCode,
        message: this.message
      })
    }
  }

  buildHtml (template) {
    return {
      statusCode: this.statusCode,
      headers: {
        'Content-Type': 'text/html'
      },
      body: template.replace('{{ message }}', this.message)
    }
  }

  buildRedirect (redirectUrl) {
    return {
      statusCode: this.statusCode,
      headers: {
        'Content-Type': 'text/plain',
        'Location': redirectUrl
      },
      body: this.message
    }
  }
}

To make use of the above class you would first create an instance of it by passing in a status code and message, you would then call one of the three public methods according to the desired response.

For example, to build a JSON a response you would do:

const statusCode = 200
const message = 'Form submission successfully made'
const response = new Response(statusCode, message)
callback(null, response.buildJson())

By default a HTML response is shown by with a generic success message:

This page is generated by loading a local HTML template file and doing a find and replace on a message variable. The template is loaded in the handler using the Node.js FS module:

const path = path.resolve(__dirname, 'template.html')
const template = fs.readFileSync().toString()

const message = 'Form submission successfully made'
const html = template.replace('{{ message }}', message)

Determining the response type

Response is responsible for building the responses, but it has no concept of what the appropriate response is. The choice of the response is determined by a series of conditionals that look at the request validated earlier in the handler.

if (request.redirectUrl) {
  callback(null, response.buildRedirect(request.redirectUrl))
  return
}

if (request.responseFormat === 'json') {
  callback(null, response.buildJson())
  return
}

if (request.responseFormat === 'html') {
  const path = path.resolve(__dirname, 'template.html')
  const template = fs.readFileSync(path).toString()
  callback(null, response.buildHtml(template))
  return
}

If you weren’t aware, you can return undefined in JavaScript by not specifying a return value. In the code block above we’re using it to stop the execution so that callback is only ever invoked once.

Structuring the receive handler

Building again on the ability to sequentially chain promises, the receive handler takes the following format:

module.exports.handle = (event, context, callback) => {
  const request = new Request(event)

  request.validate()
    .then(function () {
      // send email
    })
    .then(function () {
      // build success response
    })
    .catch(function (error) {
      // build error response
    })
    .then(function (response) {
      // response callback
    })
}

Objects built using Response are passed to callback to return a HTTP response in the final success handler on this top-level promise chain. This response could have been resolved from either the previous then() or catch() callback. Replacing comments with implementation code we arrive at the finalised handler:

module.exports.handle = (event, context, callback) => {
  const request = new Request(event)
  request.validate()
    .then(function () {
      const payload = {
         recipients: request.recipients,
         userParameters: request.userParameters
      }
      return aws.invokeLambda('formplug', 'dev', 'send', payload)
    })
    .then(function () {
      const statusCode = request.redirectUrl ? 302 : 200
      const message = 'Form submission successfully made'
      const respnse = new Response(statusCode, message)
      return Promise.resolve(response)
    })
    .catch(function (error) {
      const response = new Response(error.statusCode, error.message)
      return Promise.resolve(response)
    })
    .then(function (response) {
      if (request.redirectUrl) {
        callback(null, response.buildRedirect(request.redirectUrl))
        return
      }
      if (request.responseFormat === 'json') {
        callback(null, response.buildJson())
        return
      }
      if (request.responseFormat === 'html') {
        const path = path.resolve(__dirname, 'template.html')
        const template = fs.readFileSync(path).toString()
        callback(null, response.buildHtml(template))
        return
      }
    })
}

Invoking the send Lambda

Not discussed is how emails are actually sent. Emails get sent using Amazon Simple Email Service (SES) in the send Lambda, which is invoked in the first promise success handler in the receive Lambda. It’s invoked with a payload containing the recipients and form inputs:

const payload = {
 recipients: request.recipients,
 userParameters: request.userParameters
}

return aws.invokeLambda('formplug', 'dev', 'send', payload)

The Lambda is invoked using the AWS SDK available on NPM. Although it might look as though we’re directly calling the AWS SDK, we’re not. We’re actually calling a method on a singleton instance of a wrapping class. It provides a proxy to the library, exposing only the relevant methods in a simpler API.

const aws = require('aws-sdk')

class AwsService {
  constructor (aws) {
    this.aws = aws
  }

  invokeLambda (serviceName, stage, functionName, payload) {
    let event = {
      FunctionName: `${serviceName}-${stage}-${functionName}`,
      InvocationType: 'Event',
      Payload: JSON.stringify(payload)
    }
    return new this.aws.Lambda().invoke(event).promise()
  }

  sendEmail (email) {
    return new this.aws.SES().sendEmail(email).promise()
  }
}

module.exports = new AwsService(aws)

Sending emails

The send Lambda first creates an instance of Email and then calls sendEmail on the previously described AwsService class. In this handler, the event argument is just the payload from the receive Lambda.

module.exports.handle = (event, context, callback) => {
  const email = new Email(config.SENDER_ARN, config.MSG_SUBJECT)
  email.build(event.recipients, event.userParameters)
    .then(function (email) {
      return aws.sendEmail(email)
    })
    .catch(function (error) {
      callback(error)
    })
}

Configuration variables are loaded from a local JSON file. The SENDER_ARN variable is the Amazon Resource Name of the sending email address (SES only sends emails from verified email addresses).

const config = require('./config.json')

The build method on Email class creates an SES compatible object. It first checks that SENDER_ARN is valid and then returns the SES object.

build (recipients, userParameters) {
  return this._validateArn()
    .then(() => {
      let email = {
        Source: this._buildSenderSource(),
        ReplyToAddresses: recipients.replyTo,
        Destination: {
          ToAddresses: [recipients.to],
          CcAddresses: recipients.cc,
          BccAddresses: recipients.bcc
        },
        Message: {
          Subject: {
            Data: this.subject
          },
          Body: {
            Text: {
              Data: this._buildMessage(userParameters)
            }
          }
        }
      }

      return Promise.resolve(email)
    })
}

The body is built by looping over the user parameters that were sent as part of event. Formplug configuration variables prefixed by an underscore are omitted.

_buildMessage (userParameters) {
  return Object.keys(userParameters)
    .filter(function (param) {
      // don't send private variables
      return param.substring(0, 1) !== '_'
    })
    .reduce(function (message, param) {
      // uppercase the field names and add each parameter value
      message += param.toUpperCase()
      message += ': '
      message += userParameters[param]
      message += '\r\n'
      return message
    }, '')
}

Wrapping up

I hope you found this high-level codebase walkthrough interesting. For more information and for deployment instructions, you should go check out the repository on Github. If you have any comments or suggestions, please raise an issue or drop me an email and I’ll happily get back to you.

The cover image for this post uses graphics from SAP Scenes.

Using Google Sheets and Google Apps Script to build a blog CMS

Using Google Sheets and Google Apps Script to build a blog CMS

How to build a Serverless URL shortener using AWS Lambda and S3

How to build a Serverless URL shortener using AWS Lambda and S3