Authentication error after migrating to forest-express-sequelize v7

Expected behavior

Access Forest Admin after migrating to forest-express-sequelize@7

Actual behavior

When trying to access the Development environment with v7 installed I’m seeing an error “Please verify that your admin backend is correctly configured and running and that you have access to the internet.” Staging and Production environments that have v6 installed work fine.

Call to https://api.forestadmin.com/oidc/auth includes a correct clientId and returns a proper callback url with a code. Call to callback url fails.

Failure Logs

app_1       | POST /admin/forest/authentication 200 1056 - 2.647 ms
app_1       | OPTIONS /admin/forest/authentication/callback?code=XXXXXXXXXXXXXXX&state=%7B%22renderingId%22%3A76844%7D 204 0 - 0.258 ms
app_1       | [Nest] 31   - 05/06/2021, 11:11:21 AM   [ExceptionsHandler] Forest cannot authenticate the user for this request. +368092ms
app_1       | GET /admin/forest/authentication/callback?code=XXXXXXXXXXXXXXX&state=%7B%22renderingId%22%3A76844%7D 500 52 - 2.755 ms

Context

I’ve done everything that is listed in migration guide, including:

  • set APPLICATION_URL=http://localhost:8000/admin
  • updated CORS setup as described in the migration guide
  • made a POST request to https://api.forestadmin.com/oidc/reg with the following body:
{
	"token_endpoint_auth_method": "none",
	"redirect_uris":["http://localhost:8000/admin/forest/authentication/callback"]
}

using FOREST_ENV_SECRET as a Bearer token

  • received the following response and set FOREST_CLIENT_ID env variable using returned client_id
{
  "token_endpoint_auth_method": "none",
  "redirect_uris": [
    "http://localhost:8000/admin/forest/authentication/callback"
  ],
  "application_type": "web",
  "grant_types": [
    "authorization_code"
  ],
  "response_types": [
    "code"
  ],
  "environment_id": 64246,
  "client_id": "........................................................."
}

Versions

  • Package Version: forest-express-sequelize@7.7.0
  • Express Version: 4.17.1
  • Sequelize Version:
  • Database Dialect:
  • Database Version:
  • Project Name: Kick App

Hello @agarbund, welcome in the community!

I suppose that you are not running multiple processes of your development instances at the same time, so you don’t need to manually call the registration endpoint. The FOREST_CLIENT_ID will be generated on the fly on the first authentication request.

I don’t think that it will fix your issue, but it’s worth trying.

Is it possible to open the developer tools in your browser before accessing to your liana, in order to log network accesses? Then please find the call that is failing, and paste here all the info about this request (request & response headers & body).

It’ll help identifying the issue

Thanks

Hello @GuillaumeGautreau Thanks for a quick reply!

After removing FOREST_CLIENT_ID env variable I’m receiving a following error on backend when calling /forest/authentication

app_1       | [forest] 🌳🌳🌳  Unable to register the client
app_1       | {
app_1       |   "configuration": {
app_1       |     "authorization_endpoint": "https://api.forestadmin.com/oidc/auth",
app_1       |     "device_authorization_endpoint": "https://api.forestadmin.com/oidc/device/auth",
app_1       |     "claims_parameter_supported": false,
app_1       |     "claims_supported": [
app_1       |       "sub",
app_1       |       "email",
app_1       |       "sid",
app_1       |       "auth_time",
app_1       |       "iss"
app_1       |     ],
app_1       |     "code_challenge_methods_supported": [
app_1       |       "S256"
app_1       |     ],
app_1       |     "end_session_endpoint": "https://api.forestadmin.com/oidc/session/end",
app_1       |     "grant_types_supported": [
app_1       |       "authorization_code",
app_1       |       "urn:ietf:params:oauth:grant-type:device_code"
app_1       |     ],
app_1       |     "id_token_signing_alg_values_supported": [
app_1       |       "HS256",
app_1       |       "RS256"
app_1       |     ],
app_1       |     "issuer": "https://api.forestadmin.com",
app_1       |     "jwks_uri": "https://api.forestadmin.com/oidc/jwks",
app_1       |     "registration_endpoint": "https://api.forestadmin.com/oidc/reg",
app_1       |     "response_modes_supported": [
app_1       |       "query"
app_1       |     ],
app_1       |     "response_types_supported": [
app_1       |       "code",
app_1       |       "none"
app_1       |     ],
app_1       |     "scopes_supported": [
app_1       |       "openid",
app_1       |       "email",
app_1       |       "profile"
app_1       |     ],
app_1       |     "subject_types_supported": [
app_1       |       "public"
app_1       |     ],
app_1       |     "token_endpoint_auth_methods_supported": [
app_1       |       "none"
app_1       |     ],
app_1       |     "token_endpoint_auth_signing_alg_values_supported": [],
app_1       |     "token_endpoint": "https://api.forestadmin.com/oidc/token",
app_1       |     "request_object_signing_alg_values_supported": [
app_1       |       "HS256",
app_1       |       "RS256"
app_1       |     ],
app_1       |     "request_parameter_supported": false,
app_1       |     "request_uri_parameter_supported": true,
app_1       |     "require_request_uri_registration": true,
app_1       |     "claim_types_supported": [
app_1       |       "normal"
app_1       |     ]
app_1       |   },
app_1       |   "registration": {
app_1       |     "redirect_uris": [
app_1       |       "http://localhost:8000/admin/forest/authentication/callback"
app_1       |     ],
app_1       |     "token_endpoint_auth_method": "none"
app_1       |   },
app_1       |   "error": {
app_1       |     "name": "RequestError",
app_1       |     "code": "ECONNREFUSED",
app_1       |     "timings": {
app_1       |       "start": 1620301581680,
app_1       |       "socket": 1620301581683,
app_1       |       "lookup": 1620301581685,
app_1       |       "error": 1620301581686,
app_1       |       "phases": {
app_1       |         "wait": 3,
app_1       |         "dns": 2,
app_1       |         "total": 6
app_1       |       }
app_1       |     }
app_1       |   }
app_1       | }
app_1       | [forest] 🌳🌳🌳  Unexpected error: connect ECONNREFUSED 127.0.0.1:443
app_1       | {
app_1       |   "name": "RequestError",
app_1       |   "code": "ECONNREFUSED",
app_1       |   "timings": {
app_1       |     "start": 1620301581680,
app_1       |     "socket": 1620301581683,
app_1       |     "lookup": 1620301581685,
app_1       |     "error": 1620301581686,
app_1       |     "phases": {
app_1       |       "wait": 3,
app_1       |       "dns": 2,
app_1       |       "total": 6
app_1       |     }
app_1       |   },
app_1       |   "stack": "RequestError: connect ECONNREFUSED 127.0.0.1:443\n    at ClientRequest.<anonymous> (/app/admin/node_modules/got/dist/source/core/index.js:956:111)\n    at Object.onceWrapper (events.js:422:26)\n    at ClientRequest.emit (events.js:327:22)\n    at ClientRequest.origin.emit (/app/admin/node_modules/@szmarczak/http-timer/dist/source/index.js:39:20)\n    at TLSSocket.socketErrorListener (_http_client.js:426:9)\n    at TLSSocket.emit (events.js:315:20)\n    at emitErrorNT (internal/streams/destroy.js:92:8)\n    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)\n    at processTicksAndRejections (internal/process/task_queues.js:84:21)\n    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)"
app_1       | }

Request headers:

POST /admin/forest/authentication HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Content-Length: 23
Pragma: no-cache
Cache-Control: no-cache
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36
Content-Type: application/json; charset=utf-8
Accept: */*
Origin: http://app.forestadmin.com
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://app.forestadmin.com/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

Request payload:

{"renderingId":"76844"}

Response headers:

HTTP/1.1 500 Internal Server Error
X-Protected-By: Sqreen
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-DNS-Prefetch-Control: off
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Download-Options: noopen
X-Powered-By: Express
Access-Control-Allow-Origin: http://app.forestadmin.com
Vary: Origin
Access-Control-Allow-Credentials: true
Content-Type: application/json; charset=utf-8
Content-Length: 95
ETag: W/"5f-f38xyTWwHTfC5Wzl2VaRpthnnTI"
Date: Thu, 06 May 2021 11:46:21 GMT
Connection: keep-alive

Hello,

Your agent needs to be able to connect to api.forestadmin.com for multiple purposes. The first case is when registering (to get a client id), but also to get users’ rights.

It seems that there is some kind of DNS filtering that redirects api.forestadmin.com to localhost if I understand the error message.

Can you check that you can curl/wget https://api.forestadmin.com/ from this machine? You should get a 404 and not a connection error.

I confirmed I can access api.forestadmin.com from this machine.

adrian in ~/develop/juice-app/backend on branch master > curl https://api.forestadmin.com/
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /</pre>
</body>
</html>

In the end I can just put the FOREST_CLIENT_ID env variable manually, as I did at the beginning, but still running into authentication issue then.

Hello,

I think that the error on the client id is a symptom of the real issue. So it won’t help to fix the value in your case.

How are you running your agent? Are you directly running npm start from your command line?

The error here is pretty clear: connect ECONNREFUSED 127.0.0.1:443, the agent is trying to connect to https://localhost when trying to register the client.

It seems that your agent is trying to connect to 127.0.0.1 instead of api.forestadmin.com. Setting a static client ID won’t help because your agent needs to be able to connect to api.forestadmin.com for the rest of authentication.

Can you check that you did not specify a value for the environment variable FOREST_URL?

Ok. I’ve managed to solve the issue with not being able to fetch clientId and now I’m back into issue with callback url not working. It seems to be caused by the fact that I have my admin panel hosted on /admin path and not the root url.

When Forest is available on http://localhost:8000/admin and APPLICATION_URL env variable is set to the same url, a call to authorization callback url returns 401. Please see the logs below:

http://localhost:8000/admin/forest/authentication/callback?code=PAp63joBSpWBDtfPgKHSlIS8JXr2Ux8AZOfksdqQ7HEwRM7aGnRJHC2Knj-85tpV&state=%7B%22renderingId%22%3A76844%7D

Request headers:

GET /admin/forest/authentication/callback?code=PAp63joBSpWBDtfPgKHSlIS8JXr2Ux8AZOfksdqQ7HEwRM7aGnRJHC2Knj-85tpV&state=%7B%22renderingId%22%3A76844%7D HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36
Content-Type: application/json; charset=utf-8
Accept: */*
Origin: https://app.forestadmin.com
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

Response headers:

HTTP/1.1 401 Unauthorized
X-Powered-By: Express
Access-Control-Allow-Origin: https://app.forestadmin.com
Vary: Origin
Access-Control-Allow-Credentials: true
Content-Security-Policy: default-src 'none'
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=utf-8
Content-Length: 142
Date: Fri, 07 May 2021 12:13:11 GMT
Connection: keep-alive

Response:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>[object Object]</pre>
</body>
</html>

When I remove the /admin path all configs and put the admin interface on root url, everything works correctly.

The setup with /admin path was working correctly in forest-express-sequelize v6

Ok, can you share with us how you solved your first issue? It might help other people having the same kind of error.

Can you share with us your configuration and code that is stating that forest-express-sequelize should respond to the route /admin?

So, my current setup is the following: I have a Nest.js application, which takes all the routing from ForestAdmin and plugs it under /admin path (Nest.js is using express too). As I discovered, if I’ll create a fresh new Express instance (without Nest.js) and connect ForestAdmin routing to it, then ECONNRESET error during client id retrieval is gone.

This is not related to ForestAdmin then, but rather some issue in how Nest.js wraps Express and I’ll keep investigating it (in the end I can just use FOREST_CLIENT_ID env variable).

However, even with plain Express used, I’m experiencing authorization error in the later stage.

This is my main Express config (index.js):

const express = require('express');
const admin = require('../juice-app/backend/admin/app.js');

const app = express();

app.use('/admin', admin);

app.listen(8000);

And this is included app.js, which was generated by Lumber some time ago and updated for v7 as described in the migration guide:

const express = require('express');
const requireAll = require('require-all');
const path = require('path');
const cookieParser = require('cookie-parser');
const bodyParser = require('body-parser');
const cors = require('cors');
const jwt = require('express-jwt');
const morgan = require('morgan');
const {
  ensureAuthenticated,
  PUBLIC_ROUTES,
} = require('forest-express-sequelize');

// also tried: const app = express.Router(); but no difference
const app = express();

app.use(morgan('tiny'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));

let allowedOrigins = [/\.forestadmin\.com$/, /localhost:\d{4}$/ ];

if (process.env.CORS_ORIGINS) {
  allowedOrigins = allowedOrigins.concat(process.env.CORS_ORIGINS.split(','));
}

const corsConfig = {
  origin: allowedOrigins,
  allowedHeaders: ['Authorization', 'X-Requested-With', 'Content-Type'],
  maxAge: 86400, // NOTICE: 1 day
  credentials: true,
};

app.use('/forest/authentication', cors({
  ...corsConfig,
  origin: corsConfig.origin.concat('null')
}));

app.use(cors(corsConfig));

app.use(jwt({
  secret: process.env.FOREST_AUTH_SECRET,
  credentialsRequired: false,
}));

app.use('/forest', (request, response, next) => {
  if (PUBLIC_ROUTES.includes(request.url)) {
    return next();
  }
  return ensureAuthenticated(request, response, next);
});

requireAll({
  dirname: path.join(__dirname, 'routes'),
  recursive: true,
  resolve: Module => app.use('/forest', Module),
});

requireAll({
  dirname: path.join(__dirname, 'middlewares'),
  recursive: true,
  resolve: Module => Module(app),
});

module.exports = app;

and here is the Liana initialization:

const chalk = require('chalk');
const path = require('path');
const Liana = require('forest-express-sequelize');
const { objectMapping, connections } = require('../models');

module.exports = async function (app) {
  app.use(await Liana.init({
    configDir: path.join(__dirname, '../forest'),
    envSecret: process.env.FOREST_ENV_SECRET,
    authSecret: process.env.FOREST_AUTH_SECRET,
    objectMapping,
    connections,
  }));

  console.log(chalk.cyan('Your admin panel is available here: https://app.forestadmin.com/projects'));
};

The app is started with the following command:

APPLICATION_URL=http://localhost:8000/admin DATABASE_URL=postgres://postgres@postgres:5432/juice_app DATABASE_SCHEMA=public DATABASE_SSL=false DATABASE_ENCRYPT=false FOREST_AUTH_SECRET=xxxxxxxxxxxxxxxx FOREST_ENV_SECRET=xxxxxxxxxxxxxxxxxxx node index.js

And it breaks (returns 401) when doing a call to authorization callback url ( http://localhost:8000/admin/forest/authentication/callback?code=xxxxxxxxxxxxxxxxxxxxxx&state=%7B%22renderingId%22%3A76844%7D )

When I remove /admin path but keep wrapping ForestAdmin into external express instance everything works like it should.

So my working index.js:

const express = require('express');
const admin = require('../juice-app/backend/admin/app.js');

const app = express();

app.use('/', admin);

app.listen(8000);

Running command:

APPLICATION_URL=http://localhost:8000 DATABASE_URL=postgres://postgres@postgres:5432/juice_app DATABASE_SCHEMA=public DATABASE_SSL=false DATABASE_ENCRYPT=false FOREST_AUTH_SECRET=xxxxxxxxxxxxxxxx FOREST_ENV_SECRET=xxxxxxxxxxxxxxxxxxx node index.js

And url in environment config in ForestAdmin panel updated to http://localhost:8000

Let me know if it’s clear or shall I attach a Github repo with reproduction

Hey @agarbund,

A reproduction repository would definitely be useful.
Also, we do have an option on liana.init, allowing to pass an existing instance of a parent express app (expressParentApp) that should mount the forest routes on top of an existing express app.

  app.use(await Liana.init({
    expressParentApp: youExistingExpressParentApp,
    configDir: path.join(__dirname, '../forest'),
    envSecret: process.env.FOREST_ENV_SECRET,
    authSecret: process.env.FOREST_AUTH_SECRET,
    objectMapping,
    connections,
  }));

Let me know if that helps & if you are able to provide a reproduction repository :pray:

Hey @jeffladiray
expressParentApp seems like sth what I need! Do you have some docs about how to use it?

Two questions regarding it:

  1. In this case, in app.use(await Liana.init()) should app be internal Forest express instance or mine express?
  2. What about all other routing configuration that takes place in app.js generated by Lumber?

I’ll provide the reproduction repo today. Will let you know.

@jeffladiray here is the reproduction repo: GitHub - agarbund/forest-authentication-issue

Hey @agarbund

I just discussed with the team and from what I understood, expressParentApp will not be helpful here.

We have some known issues with routes prefixes in v7, and I’m still discussing with the team to check if we have a known solution for this problem. I’ll update the thread once I got more feedback.

Thanks for the reproduction repo, it’ll be helpful :pray:

@jeffladiray Any update on this?

Hey @agarbund, and sorry for the delayed response.

It appear that we currently have an issue with nested app path (/admin in your case) which is causing this issue. Some of the routes we use do not currently support this route nesting, (Especially the authentication).

A ticket is open on our end but the problem associated with this nesting does not stop here and requires a bit of work to be fully functional. I might be able to provide a workaround for the authentication part if that a requirement for you. Otherwise, be sure we will update this thread upon release of a fix.

1 Like

So the issue with ECONNREFUSED error and request to localhost was caused by 3-rd party packages using agent-base package (I had to update all of them to fix the issue). The original issue with callback url not working remains.

For anyone else who’ll get here after having ECONNREFUSED error when requesting 127.0.0.1:443: run npm ls agent-base and try to update all of packages which are using agent-base package.

1 Like

Hey @jeffladiray @GuillaumeGautreau any chance the issue with authorization not working when hosting admin panel on subpath will be fixed anytime soon? Forest interface yells at me that I should already upgrade to v7.

Hello @agarbund,

I created a high-priority issue on our side here:

We will be working on it very soon.

1 Like

Hello @agarbund,

I managed to make it work with your repository and a small change:

in app.js I changed

app.use('/forest', (request, response, next) => {
  if (PUBLIC_ROUTES.includes(request.url)) {
    return next();
  }
  return ensureAuthenticated(request, response, next);
});

Into

app.use('/forest', (request, response, next) => {
  if (PUBLIC_ROUTES.includes(request.path) || request.method === "OPTIONS") {
    return next();
  }
  return ensureAuthenticated(request, response, next);
});

This code is not executed in the context of standalone agents, for public routes, but in your case it was filtering calls to public routes.

With this small change & the latest version of forest-express-sequelize, the authentication is running smoothly and the app is exposing the forest agent under /admin.

1 Like

It worked! Thank you @GuillaumeGautreau

1 Like