Migrate RoR to AWS Lambda

We are long time customers and have been using Forest Admin for years.

We have ran it with the RoR GEM so far. With Rails it works very magically. We dropped it there and now everything works fine.

We are migrating from RoR on heroku to Infrastructure-as-Code on AWS (using Serverless Stack). We will be using the same PG database but behind a RDS cluster. We won’t have anymore ruby and use a domain driven approach and build our API in TypeScript with ApiGateway and Lambdas.

My questions:

  1. Is there an out of the box magic solution like the one in ruby for Lambdas? I’m guessing you use the schema.rb and other RoR utilities to deduct the data structure

  2. What are the main components we need to take in account? Tell forest our data structure / user authentication / Forest endpoint to fetch data, etc.

  3. We will be using Kysely to type our database in Typescript, I’m not sure if this is something useful to mention.

There are two issue with running a forest admin agent in AWS lambda.

The first one is that requests from the internet need to reach the agent itself.

This would be solvable by using API Gateway + a module like serverless-express (or similar in ruby, I don’t know that ecosystem very well).

But then API Gateway comes with its own limitations (the maximum allowed payloads are quite restricted, so some features like csv export would not work)

The second one (which much harder to solve) is that there is a lot of internal state within an agent.

Each time your agent starts, it needs to retrieve from our servers a lot of information (about permissions, ip whitelist/blacklists, the list of charts that can be generated, and so on).

This can take up to several seconds.
Startup time is not an issue when you host your agent in a server, as the process runs for a long time, but that is not the case when using function-as-a-service solutions

IMHO, is you are migrating all of your code to typescript, a better solution would be using our new agent in TypeScript, which is in beta, and will be officialy released very shortly

Hosting that using Fargate, can be a good solution if you no longer want to manage your infrastructure like you did before.

Here is the doc about that new agent: Forest Admin - [Beta] Developer guide

Ok, well, we don’t really mind the as long as we can deploy and automate it with AWS CDK.

But essentially, it should be possible?

The part I’m not sure is how to generate that forest schema and give it to the app? Should we have it from our PG database directly? Can we compute it from our TS Types / GQL types?

Hosting in lambda would not be possible because of the points I mentioned

  • the agent is not stateless
  • there are restrictions on payload sizes with lambda

Sorry, I wasn’t clear enough.

In theory hosting this on some AWS service (fargate for example) and having it IaC should be possible.

One part I’m still not on point from my previous post:

How to generate that forest schema and give it to the app? Should we have it from our PG database directly? Can we compute it from our TS Types / GQL types?

If you use the new typescript agent with the @forestadmin/datasource-sql connector, the database is automatically introspected when the agent starts, so you don’t need to provide any file so that we understand the structure of the db.

If you want, you can try it out on a test project!

Currently this is achieved by choosing “NestJS” agent during the onboarding (as we’re in beta), and then NOT using the sample of code which is provided at onboarding, and using this one instead:

and then connecting the agent to your database:

This should give you a minimal agent working with very little code (an no need to provide the structure of the db)

Thanks for the help on this thread! It definitely pointed me in the right direction, namely don’t use Lambdas.

But I was able to get the Forest Admin backend agent running in NodeJS/express/sequelize on Fargate, and served the endpoints through an HTTP API Gateway. We open sourced the setup as a reference architecture for anyone who might be trying to use a similar deployment model! It uses Terraform to manage the AWS services, and Docker to package the NodeJS app. (We declare models in code for Sequelize, but it should be able to do self-discovery with a direct db link according to FA docs.)

https://github.com/jumpwire-ai/infrastructure-tools/tree/main/terraform/aws/ecs/example/nodejs

2 Likes