Most tables fail to load in production, staging is fine

Feature(s) impacted

Visibility of my data

Observed behavior

I got a layout working nicely in staging, then I released to production. The one in staging works perfectly, the one in production loads but will only show 2 or 3 of my tables. The other tables give server 400 errors.

The backend services for staging and production are currently different API deployments, but I know that the backend is exactly the same, because I run it as a containerized image. The same image version is running in both environments.

I checked the logs for the API in production, there are no errors reported in the logs. This makes it difficult to debug. It’s also the case that the documentation states that you can only turn on debugging by setting environment to development, which I can’t do for the production backend - right?

Expected behavior

After releasing to production, production should work exactly like staging.

Failure Logs

server logs here, note there is no error reported

Context

I really want to love ForestAdmin, but the per-seat cost is high and the platform’s tools for wiring in the backend is a bit confusing and difficult to use. I’m hanging with you guys for now though to see if we can get over the many hurdles we’ve faced (broken merges, no rollback or versioning, hard to debug with available documentation, and an odd-feeling ‘community support’ paradigm for paying users). I hope this is going to work out. Let me know if there’s something I’m missing that would make this all easier.

  • Project name: bluebot-customer-support-management
  • Team name: bluebot
  • Environment name: production
  • Agent technology: nodejs
  • Agent (forest package) name & version:
    @forestadmin/agent”: “^1.0.0”,
    @forestadmin/datasource-sql”: “^1.0.0”,
  • Database type: postgres
  • Recent changes made on your end if any: pushed to production

Hi @ZavenArra,

Sorry to ear you got issue.
Could you please shared the full error? I see you have another error before the one you shared.
Could you also share the response you get from the server?
Is it a CORS issue?

Hi Vince,

Doesn’t your frontend have a way to log errors on my paid account, so your support team can help me out a bit quicker in the production environment so we can rely on it?

Here is the response from the server

{
    "errors": [
        {
            "name": "ValidationError",
            "detail": "Invalid projection: field not found 'feature.module'}",
            "status": 400
        }
    ]
}

There is no field feature.module in our schema. The field is called module_name. We are running the same version of the backend API in staging and production, and the version of our database schema, yet staging works fine while production does not.

We don’t store clients data.
Even errors can expose clients data, so they are not exposed on our side, that’s why I’m asking.

In this case, it looks clearly like a server error.
Let me check if I can find anything to help you :male_detective:

Do you define yourself your models with Sequelize or are you using the createSqlDataSource function ?

Regarding storage of clients data…

Since you only supply support via a public message board, you have forced me to expose the error not just to your trusted team, whom I pay, but to the entire internet.

Does it occur to your team that this pattern of support defeats the very purpose ? Do you have a secure channel that we can continue this communication so that I can stop exposing my implementation to hackers who may peruse this forum, to look at logs posted here to expose vulnerabilities in your system, or allow attackers to gather domain specific knowledge that could be used in social engineering attacks?

The models were auto-generated using your CLI, I did not do any manual definition of the modules in the typings.js file. I searched and found where module_name is used in the auto-generated file, and it is correct. The database has a field called feature.module_name. Why does this work in development but not production - the API version and database schema are the same? If I am supposed to debug this on my side, can you give me any further guidance about how the internals of your node.js agents work and how I would go about debugging a problem this like this?

Could you check that your .forestadmin-schema.json is the same on both your environments ?
If you could share your models definition it could also help. specially the feature and module models?
If you are to afraid to share your data you can still send it to me in private message :wink:

Otherwise you can plan a video meeting here with our CS team Calendly - Salim Ameur

I’m having the same problem. I’m using AWS Elastic Container Service and there have been no problems so far. Did you solve the problem?

@ZavenArra

Even though I haven’t made any changes in the last month, forest is not working today.

Hey @Shape_Fit,

Could you share your errors and how you created your agent?
Also it would really help if you could let me know on which version you are running your backend?
Do you have an environment that is working?