How to configure ForestAdmin with a nextjs backend server

Hi Forest community!

I have a backend server using the Nextjs framework. To be more precise I use their API routes feature, which means each of my api routes is served as a serverless function by Vercel. So I don’t have a classic backend server up and running all the time, and I was wondering if Forest Admin is compatible with this type of serverless architecture.

I can create api routes to handle actions in forest admin panel, like smart actions, but how could I init the agent, and make the .forestadmin-schema.json and all the config visible by Forest ?

(I saw an old post on the same topic, but the answer mentioned NestJS instead of NextJS)

Thanks for your help

Hello @Jeremy_Fuhrmann,

Although we have been able to make the Forest Admin agent work with severless architectures, we are sadly not compatible with Vercel as it does not support Sequelize which is the ORM we rely on. You can find more info on this thread in reddit. Note that our team tried to integrate with Vercel during this summer and it was still the case.

The only way would be to recode a @forestadmin/datasource integrated with Prisma which is compatible with Vercel.

Best regards,

Hi @dogan.ay,

Thank you for your answer, sorry my description was not clear: my backend server is hosted on vercel serverless functions, but our database is hosted on AWS, not on Vercel. And we are already using a Sequelize connection in our backend, so it should not be an issue.

My question was more about the serverless part, is there any specific configuration needed for this use case, compared to an express server for instance? How should we init the forest agent, and do we need to add some specific backend routes to ensure communication with forest server ?

Thanks!

There is no out of the box configuration for the agent to be run on serverless, it should be able to start normally if you host a standalone version of our agent. But you might encounter other issues related to Vercel’s default timeout and whatnot, as you will do a whole startup sequence of your agent on each invocation.

Hey :wave:

Ok so that’s a complex topic. Vercel (And serverless) being what they are, you’ll need a few tweaks in the default configuration to make this work.

To be honest, my tests are super recent and as stated by @dogan.ay, we don’t offer official support of self-hosted serverless agent. So everything below is super experimental.

I’m not sure about what type of machine will run your code, so I’ll base the following from a free tier account:

  • Vercel has a strong timeout limit, ~10s per call. Even though our server start is pretty fast, it is splitted in multiple operations that can sometimes being time consuming, especially on slow machines
    • On server start, we connect the the database (If you database requires SSL, the is the SSL/TLS negociation)
    • We run a bunch of SQL queries to get your overall schema
    • We generate a .forestadmin-schema.json

All of these, at least from my test on a free tier machine, takes more than 8/9s on a medium sized database (~12 tables, ~100k rows per table).

To mitigate these issues:

  • Sadly, not so much can be done on the SSL/TLS & connection aspect. From my test, there is a bit of time that could be reduced here, but I didn’t push my tests further.
  • We can drastically reduce time on the introspection step, by just not doing it on vercel. See in the code below.
  • Same goes for the schema. We can avoid regenerating the schema by using isProduction: true, and we can avoid sending it (Kind of error prone though) by using skipSchemaUpdate: true and finding a way to publish the schema using another mecanism (Like forest schema:apply, or using a CI process to publish the schema to our servers).

Overall, I’m running the following base code (No customizations)

index.ts

import type { SslMode } from '@forestadmin/datasource-sql';

import 'dotenv/config';
import { createAgent } from '@forestadmin/agent';
import { createSqlDataSource, introspect } from '@forestadmin/datasource-sql';
import pg from 'pg';
import fs from 'fs';
import express from 'express';
import type { Schema } from './typings';

async function buildServer() {
  const app = express();
  const dbCredentials = {
    uri: process.env.DATABASE_URL,
    schema: process.env.DATABASE_SCHEMA,
    sslMode: process.env.DATABASE_SSL_MODE as SslMode,
  };

  let introspection;
  try {
    introspection = JSON.parse(
      fs.readFileSync(`${__dirname}/my-database-introspection.json`, 'utf-8'),
    );
  } catch (e) {
    // We should write the file on disk if it doesn't exist.
    fs.writeFileSync(
      'introspection.json',
      JSON.stringify(await introspect(dbCredentials), null, 2),
    );
  }

  const agent = createAgent<Schema>({
    authSecret: process.env.FOREST_AUTH_SECRET!,
    envSecret: process.env.FOREST_ENV_SECRET!,

    // We are forced to be in production mode in Vercel, as the FS is readonly so schema cannot be written
    isProduction: true,
    schemaPath: `${__dirname}/.forestadmin-schema.json`,
    // We need to remove the typings generation on Vercel, again as FS is readonly
    // typingsPath: './typings.ts',
    // typingsMaxDepth: 5,
    instantCacheRefresh: false,
    // This is kind of touchy, as this means that the schema won't be sent. In my case, I "tricked" the system to publish my schema on a Production environment from my dev env, but you could rely on our `forest schema:apply` command.
    skipSchemaUpdate: true,
  });

  agent.addDataSource(
    createSqlDataSource(
      {
        ...dbCredentials,
        dialect: 'pg',
        dialectModule: pg,
      },
      { introspection },
    ),
  );

  // I'm using an express server here, but you should be able to use mountOnStandAloneServer
  agent.mountOnExpress(app);

  await agent.start();
  return app;
}

export default buildServer()
  .then(app => {
    app.listen(3310, () => {
      console.log('Your admin panel is available here');
    });
    return app;
  })
  .catch(console.log);

vercel.json

{
  "version": 2,
  "builds": [
    {
      "src": "index.ts",
      "use": "@vercel/node"
    },
    {
      "src": "customizations/*.ts",
      "use": "@vercel/node"
    },
    {
      "src": ".forestadmin-schema.json",
      "use": "@vercel/static"
    },
    {
      "src": "my-database-introspection.json",
      "use": "@vercel/static"
    },
    {
      "src": "typings.ts",
      "use": "@vercel/node"
    }
  ],
  "devCommand": "yarn start",
  "framework": null,
  "installCommand": "yarn install",
  "routes": [
    {
      "src": "/(.*)",
      "dest": "index.ts",
      "methods": [
        "GET",
        "POST",
        "PUT",
        "PATCH",
        "DELETE",
        "OPTIONS"
      ],
      "headers": {
          "Access-Control-Allow-Origin": "https://app.forestadmin.com",
          "Access-Control-Allow-Methods": "GET, POST, PATCH, PUT, DELETE, OPTIONS",
          "Access-Control-Allow-Headers": "Authorization, Content-Type, X-CSRF-Token, X-Requested-With, Accept, Accept-Version, Content-Length, Content-MD5, Date, X-Api-Version",
          "Access-Control-Allow-Credentials": "true"
      }
    }
  ]
}

This gives me a working standalone agent, even though obviously response time are a bit higher than what we’re use to in a stateful deployment (Between 100ms up to 3s, P95 ~900ms).

Regarding the NextJS aspect, having more informations regarding what you want to achieve would be useful. As stated by @dogan.ay, we do not offer anything specific regarding integrating directly our agent with NextJS. However, depending on your answer, you may be able to forward calls to your main app via HTTP through our Action API or Field API.

In any case, I would highly suggest to still deploy in a stateful manner whenever possible, but if you really want to make it work, the code above should allow you to run a serverless agent on vercel.

Let me know if that helps :pray:

1 Like