Issue between `signalfx-tracing` and `forest-express-moongose` sass

Vishnu_Gupta · May 31, 2022, 1:27pm

@GuillaumeGautreau We are using signalfx-tracing and forest-express-moongose in our project. Now I am getting the below issue. Found a thread related to the same issue. Can you please help me how can I make some changes in forest-express-moongose as a work around. Won’t getting any help from signalfx-tracing` authors, they suggested an alternative to use open telemetry which we don’t want to use for now.

[forest] 🌳🌳🌳  Unable to register the client
{
  "configuration": {
    "authorization_endpoint": "https://api.forestadmin.com/oidc/auth",
    "device_authorization_endpoint": "https://api.forestadmin.com/oidc/device/auth",
    "claims_parameter_supported": false,
    "claims_supported": [
      "sub",
      "email",
      "sid",
      "auth_time",
      "iss"
    ],
    "code_challenge_methods_supported": [
      "S256"
    ],
    "end_session_endpoint": "https://api.forestadmin.com/oidc/session/end",
    "grant_types_supported": [
      "authorization_code",
      "urn:ietf:params:oauth:grant-type:device_code"
    ],
    "id_token_signing_alg_values_supported": [
      "HS256",
      "RS256"
    ],
    "issuer": "https://api.forestadmin.com",
    "jwks_uri": "https://api.forestadmin.com/oidc/jwks",
    "registration_endpoint": "https://api.forestadmin.com/oidc/reg",
    "response_modes_supported": [
      "query"
    ],
    "response_types_supported": [
      "code",
      "none"
    ],
    "scopes_supported": [
      "openid",
      "email",
      "profile"
    ],
    "subject_types_supported": [
      "public"
    ],
    "token_endpoint_auth_methods_supported": [
      "none"
    ],
    "token_endpoint_auth_signing_alg_values_supported": [],
    "token_endpoint": "https://api.forestadmin.com/oidc/token",
    "request_object_signing_alg_values_supported": [
      "HS256",
      "RS256"
    ],
    "request_parameter_supported": false,
    "request_uri_parameter_supported": true,
    "require_request_uri_registration": true,
    "claim_types_supported": [
      "normal"
    ]
  },
  "registration": {
    "redirect_uris": [
      "http://localhost:3010/forest/authentication/callback"
    ],
    "token_endpoint_auth_method": "none"
  },
  "error": {
    "name": "RequestError",
    "code": "ECONNREFUSED",
    "timings": {
      "start": 1653994037273,
      "socket": 1653994037274,
      "lookup": 1653994037274,
      "error": 1653994037275,
      "phases": {
        "wait": 1,
        "dns": 0,
        "total": 2
      }
    }
  }
}
[forest] 🌳🌳🌳  Unexpected error: connect ECONNREFUSED 127.0.0.1:443
{
  "name": "RequestError",
  "code": "ECONNREFUSED",
  "timings": {
    "start": 1653994037273,
    "socket": 1653994037274,
    "lookup": 1653994037274,
    "error": 1653994037275,
    "phases": {
      "wait": 1,
      "dns": 0,
      "total": 2
    }
  },
  "stack": "RequestError: connect ECONNREFUSED 127.0.0.1:443\n    at ClientRequest.<anonymous> (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/got/dist/source/core/index.js:962:111)\n    at Object.onceWrapper (node:events:510:26)\n    at /Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/signalfx-tracing/src/scope/new/base.js:48:19\n    at Scope._activate (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/signalfx-tracing/src/scope/new/scope.js:45:14)\n    at Scope.activate (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/signalfx-tracing/src/scope/new/base.js:13:17)\n    at ClientRequest.<anonymous> (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/signalfx-tracing/src/scope/new/base.js:47:20)\n    at ClientRequest.emit (node:events:402:35)\n    at ClientRequest.emit (node:domain:475:12)\n    at ClientRequest.req.emit (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/signalfx-tracing/src/plugins/http/client.js:98:21)\n    at ClientRequest.origin.emit (/Users/vishnugupta/Projects/Reports/Lender-portal/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)\n    at TLSSocket.socketErrorListener (node:_http_client:447:9)\n    at TLSSocket.emit (node:events:402:35)\n    at TLSSocket.emit (node:domain:475:12)\n    at emitErrorNT (node:internal/streams/destroy:157:8)\n    at emitErrorCloseNT (node:internal/streams/destroy:122:3)\n    at processTicksAndRejections (node:internal/process/task_queues:83:21)\n    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1161:16)"
}

GuillaumeGautreau · May 31, 2022, 1:37pm

Hello @Vishnu_Gupta,

Thanks for your feedback.

Could you please fill in the required info about your project?

Project name: …

Agent type & version: …

Could you also try to remove signalfx-tracing from your project and check if you can make it work without it?

Vishnu_Gupta · May 31, 2022, 1:46pm

Thanks for the quick response.


* Project name : finicity-reports

* Agent type : forest-express-moongose

* Version : Liana 8.7.0

Yes, after removing signalfx-tracing it’s works fine as expected

GuillaumeGautreau · May 31, 2022, 1:50pm

Ok, then you are right, this is the same issue. Unfortunately we cannot do anything here. The library signalfx-tracing is messing around with low-level code on nodejs, and it seems to not work properly when used with got which is a library to do http requests.

The only workaround I can see is, either to deactivate signalfx on your project, or create a secondary project with forest admin, but without signalfx.

Vishnu_Gupta · May 31, 2022, 2:23pm

@GuillaumeGautreau We use to run both the sass successfully signalfx-tracing and forest-express-moongose on Liana version 6.7.2. We upgraded the forest-express-moongose package to versin 8.7.0 for fixing the vulnerability issue. Can you help us by fixing the vulnerability issue on the version 6.7.2. So we can use it until we get any updates from signalfx-tracing?

Thanks in advance

GuillaumeGautreau · May 31, 2022, 2:49pm

The library that signalfx interacts with has been introduced with the version 7 of forest-express-mongoose, that’s why upgrading to v8 caused the issue.

What kind of vulnerabilities are you trying to fix?

GuillaumeGautreau · May 31, 2022, 2:51pm

Can you try to keep signalfx-tracing but deactivate the tracing on http/https?

Vishnu_Gupta · June 2, 2022, 3:58pm

We are trying to fix the twist-lock vulnerability reported in forest-express-moongose version 6.7.2 package like moment loadash etc… Some dependencies package are reported as vulnearble.

Vishnu_Gupta · June 2, 2022, 3:59pm

It won’t be help us as we need to trace the api calls.

GuillaumeGautreau · June 3, 2022, 7:31am

It’s a bug with signalfx, and looking at the github repository of this package, it seems that it’s deprecated. You can read this in the readme section:

Deprecation Notice

The SignalFx Tracing Library for Node.js is deprecated. Only critical security fixes and bug fixes are provided.

Consider using the Splunk Distribution of OpenTelemetry JS, which offers similar capabilities and fully supports the OpenTelemetry standard. To learn how to migrate, see Migrate from the SignalFx Node.js Tracing Library.

This library is replacing some native node packages in order to intercept every external call, and it seems to have side effects with the library named got that is used by openid for http requests. This replacement/wrapping of native package is really prone to provoking errors.

Maybe we can work on reproducing the error with just signalfx and got, to see if we can spot the problem, but I’m not sure that the authors of signalfx will work on a fix except if you want to propose a PR with a fix on their repository.

If you’re OK to work on this fix on signalfx, we can provide help to reproduce the problem with got.

Vishnu_Gupta · June 21, 2022, 11:14am

@GuillaumeGautreau We have Migrate from Signalfx to Spunk Open Telemetry. Our Forest Express Mongoose to version 8.7.1 everything is working fine on my local environment. On Development environment getting error when trying to access the dashboard

Forest cannot authenticate the user for this request

Logs for your reference

[2022-06-21T10:25:09.731Z] - POST /forest/sessions-google 401 34.637 ms
[2022-06-21T10:32:39.405Z] - OPTIONS /forest/stats/Organization?timezone=Asia%2FCalcutta 204 1.727 ms
[2022-06-21T10:32:39.409Z] - OPTIONS /forest/stats/Report?timezone=Asia%2FCalcutta 204 1.282 ms
[2022-06-21T10:32:39.412Z] - OPTIONS /forest/stats/Report?timezone=Asia%2FCalcutta 204 2.148 ms
[2022-06-21T10:32:39.681Z] - POST /forest/stats/Organization?timezone=Asia%2FCalcutta 401 13.602 ms
[2022-06-21T10:32:39.689Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 5.518 ms
[2022-06-21T10:32:39.694Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 3.876 ms
[2022-06-21T10:35:57.165Z] - POST /forest/stats/Organization?timezone=Asia%2FCalcutta 401 5.043 ms
[2022-06-21T10:35:57.229Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 4.540 ms
[2022-06-21T10:35:57.260Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 4.868 ms
[2022-06-21T10:39:54.545Z] - OPTIONS /forest/sessions-google 204 2.642 ms
{
  code: '',
  message: 'Forest cannot authenticate the user for this request.'
}

Below is the code for reference.

const cors = require('cors');
const config = require('config');
const join = require('path');
const Liana = require('forest-express-mongoose');

module.exports = async function(app) {
  const mongoose = require('mongoose');

  const allowedOrigins = [/\.forestadmin\.com$/,/\.finicitydev\.com$/, /\.finicitystg\.com$/, /\.finicityreports\.com$/];
  const corsOptions = {
    origin: allowedOrigins.concat(new RegExp('null')),
    allowedHeaders: ['Authorization', 'X-Requested-With', 'Content-Type', 'Forest-Context-Url'],
    maxAge: 86400, // NOTICE: 1 day
    credentials: true,
  };
  app.use(cors(corsOptions));

  const forest = await Liana.init({
    configDir: join.resolve(__dirname, '../forest'),
    envSecret: config.FOREST_ENV_SECRET,
    authSecret: config.FOREST_AUTH_SECRET,
    objectMapping: mongoose,
    connections: { default: mongoose.connection },
  });
  app.use(forest);

  app.use('/forest', (request, response, next) => {
    if (Liana.PUBLIC_ROUTES.includes(request.path) || request.method === 'OPTIONS') {
      return next();
    }
    return Liana.ensureAuthenticated(request, response, next);
  });
  app.use('^(?!forest/?$).*', cors(corsOptions));
};

GuillaumeGautreau · June 21, 2022, 11:31am

Glad to know that the upgrade worked!

Forestadmin is requesting /forest/sessions-google which is an old route that is not implemented anymore on recent versions of forest-express-mongoose.

This indicates that Forest Admin still have the info that your environment is using an old version of forest-express-mongoose.

This info is sent by the server on startup, only if NODE_ENV is set to development. Can you please check the value of this environment variable on your machine?

There is also an environment variable named FOREST_DISABLE_AUTO_SCHEMA_APPLY that could explain this behavior, please check that you did not set this value.

Vishnu_Gupta · June 21, 2022, 1:33pm

@GuillaumeGautreau Environment variable of NODE_ENV is set to development and added environment variable named FOREST_DISABLE_AUTO_SCHEMA_APPLY value true. Now it’s did not requesting /forest/sessions-google but still getting the error check below logs

[forest] 🌳🌳🌳  Checking need for apimap update...
[forest] 🌳🌳🌳  No change in apimap, nothing sent to Forest.
[2022-06-21T13:17:29.522Z] - OPTIONS /forest/stats/Organization?timezone=Asia%2FCalcutta 204 8.555 ms
[2022-06-21T13:17:29.527Z] - OPTIONS /forest/stats/Report?timezone=Asia%2FCalcutta 204 2.602 ms
[2022-06-21T13:17:29.530Z] - OPTIONS /forest/stats/Report?timezone=Asia%2FCalcutta 204 1.526 ms
[2022-06-21T13:17:29.815Z] - POST /forest/stats/Organization?timezone=Asia%2FCalcutta 401 31.067 ms
[2022-06-21T13:17:29.828Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 10.836 ms
[2022-06-21T13:17:29.833Z] - POST /forest/stats/Report?timezone=Asia%2FCalcutta 401 3.013 ms
[2022-06-21T13:17:39.704Z] - OPTIONS /forest/sessions 204 1.543 ms
{
  code: '',
  message: 'Forest cannot authenticate the user for this request.'
}
[2022-06-21T13:17:40.763Z] - POST /forest/sessions 401 7.137 ms

GuillaumeGautreau · June 21, 2022, 1:46pm

Hello the value FOREST_DISABLE_AUTO_SCHEMA_APPLY should be removed. Please remove it.

The log indicates that the schema is sent to forest admin which should update the version on our side. Can you please give me the exact environment name you’re working on? I would like to check the value we have in the DB, and I need to be sure that we are talking about the same one.

Vishnu_Gupta · June 21, 2022, 1:57pm

@GuillaumeGautreau Below are the details

Project Name - Lender Portal
Environment - Development

GuillaumeGautreau · June 21, 2022, 2:18pm

Ok, then I still can see in our DB that the new version of forest-express-mongoose has not been received.

Can you check:

that the FOREST_ENV_SECRET on your machine ends with da
that the server you are setting up is responding to https://lender-portal.finicitydev.com
that the version 8 of forest-express-mongoose is actually installed in the directory from which the server is launched
that the logs indicate that the shema is correctly sent to Forest Admin

Thanks

Vishnu_Gupta · June 21, 2022, 4:14pm

Following items checked:

Verified - FOREST_ENV_SECRET on your machine ends with da
Verified - the logs indicate that the shema is correctly sent to Forest Admin
Verified - the version 8 of forest-express-mongoose is actually installed in the directory from which the server is launched

Just, one question https://lender-portal.finicitydev.com is only accessible in our office network. The Application URL which we passed should be public accessible?

Vishnu_Gupta · June 21, 2022, 4:47pm

@GuillaumeGautreau Just downgraded forest-express-mongoose to version 7.9.2 and everything working fine. Liana version is update properly to 7.9.2 in development environment. What may be the problem with the latest of forest-express-mongoose ?

GuillaumeGautreau · June 22, 2022, 6:46am

Hello @Vishnu_Gupta,

Did you follow the upgrade docs from v7 to v8? There is a change to be made regarding CORS, it may be related to your issue.

Vishnu_Gupta · June 23, 2022, 10:04am

Can you check what do we missed related to cors in this code.

const cors = require('cors');
const config = require('config');
const join = require('path');
const Liana = require('forest-express-mongoose');

module.exports = async function(app) {
  const mongoose = require('mongoose');

  const allowedOrigins = [/\.forestadmin\.com$/,/\.finicitydev\.com$/, /\.finicitystg\.com$/, /\.finicityreports\.com$/];
  const corsOptions = {
    origin: allowedOrigins.concat(new RegExp('null')),
    allowedHeaders: ['Authorization', 'X-Requested-With', 'Content-Type', 'Forest-Context-Url'],
    maxAge: 86400, // NOTICE: 1 day
    credentials: true,
  };
  app.use(cors(corsOptions));

  const forest = await Liana.init({
    configDir: join.resolve(__dirname, '../forest'),
    envSecret: config.FOREST_ENV_SECRET,
    authSecret: config.FOREST_AUTH_SECRET,
    objectMapping: mongoose,
    connections: { default: mongoose.connection },
  });

[/quote]

Topic		Replies	Views
Unable to register the client Help me!	17	842	June 25, 2021
Migration to v7 issue Help me!	7	615	July 10, 2021
Unable to register the client error Help me! howtos , setup	2	824	March 27, 2021
Unable to register the client error after upgrade v7 Help me!	22	785	December 2, 2022
Unable to register the client - TimeoutError Help me! setup	37	2918	December 9, 2022

Issue between `signalfx-tracing` and `forest-express-moongose` sass

Related topics