Not records found in production after first deploy

Feature(s) impacted

Finding records in the database after first deploy in production

Observed behavior

No records found in any collection.

Expected behavior

Display the records in DB

Failure Logs

No failing logs on server-side:

Only failing logs on client side dealing speaking of a trial not found

Context

the DATABASE_URL environment variable content in production is the exact same I use in my backend to reach my data.

  • Project name: slaf-forest
  • Environment name: production
  • Agent type & version:
"liana": "forest-express-mongoose",
"liana_version": "8.4.3",
"stack": {
  "database_type": "MongoDB",
  "engine": "nodejs",
  "engine_version": "16.13.0",
  "orm_version": "5.8.13"
}

Hello @guillaumejauffret

Thank you for sharing this.

For the error on the /trial endpoint, I check with the dev team and it is not linked with your issue. They’ll change this to a 200 with an empty body instead of a 404 error when you have no ongoing trial. Sorry about that.

Regarding your main point, I understand that:

  • you see data records on you development environment
  • you do not see data records on your production environment
  • both environments access the very same database, via the same URL, user and password.

Am I right?
Was that from the start? Or did you recently change this configuration?

On my end, I see not difference between both environments.

Did you try to login on the server running the production environment, copy paste the DATABASE_URL in a terminal and try to manually connect to you MongoDB server and access some records?

Thank you

hi @Guillaume_Deslandes

the first two points are right, the third is not. To be more precise regarding this third point:

  • my development environment has access to a development database
  • my production environment has access to a production database in which there is some data.

In Forest Production env, all the collections seem to be empty.

So, I’ve tried to use the production database url in development environment and know I see many warning messages such as:

[forest] 🌳🌳🌳  Cannot find the reference "transaction" on the model "File".
[forest] 🌳🌳🌳  Cannot find the reference "accountingLine" on the model "Transaction".
[forest] 🌳🌳🌳  Cannot find the reference "accountingLine" on the model "Transaction".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "accountingCategory" of your Forest customisation in "AccountingCategory".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "accountingLine" of your Forest customisation in "AccountingLine".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "bridgeApiEvent" of your Forest customisation in "BridgeApiEvent".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "bridgeUser" of your Forest customisation in "BridgeUser".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "file" of your Forest customisation in "File".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "plaidEvent" of your Forest customisation in "PlaidEvent".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "transaction" of your Forest customisation in "Transaction".
[forest] 🌳🌳🌳  DEPRECATION WARNING: Collection names are now based on the models names. Please rename the collection "user" of your Forest customisation in "User".

This may be the explanations but seems a problem to me because I don’t want to change my collection names now!

once I’ve changed collection names, Forest displays this:

@guillaumejauffret Thank you for the details.

OK, I misunderstood “the exact same” for “the same server” instead of “the same schema” :+1:

Regarding the “hidden collections” from your last post, you can edit your layout by clicking on the paint brush icon in the top right corner of your browser window.
This should activate the edit mode, and the collection list on the left panel will allow you to change the visibility of the news collections.

@Guillaume_Deslandes nop, the interface does not allow me to do this:

image

and I don’t want to change anything in my schema, model names and collection names.

Why should I change anything in my schemas to get data in production mode ?

For instance,

in my application backend, my user model is declare as so:

export const User: Model<IUser & Document> = model<IUser & Document>('User', UserSchema, 'user')
  • If I use the same declaration in my forest backend with the local database URL (such as mongodb://127.0.0.1:27017/slaf_dev) or the production database URL, the collection is not even detected by Forest.

  • If I use ('user', UserSchema, 'user') with a local database URL: I can see the user collection and its records, but with a production database URL: Forest detects the collection but says there is no record.

image

Here you can see that my collections are lowercased in my production database:

image
…meaning I must use lowercase collection name in my schemas and Forest should get the records

Hey @guillaumejauffret :wave:

I just tried on my end but I wasn’t to reproduce your issue with a forest-cli generated project. It seems like you are using typescript, am I right? Could you eventually share a complete definition of one of your model (If you consider it private, you can share it via DM)

Are you using a specific mongodb provider, so I can give this a shot with a setup similar to yours?

Thanks in advance

@jeffladiray yes I do use typescript. I just use Mongoose if it’s the question about the provider.

here is my user model schema:

import { Document, model, Model, Schema } from 'mongoose'
import beautifyUnique from 'mongoose-beautiful-unique-validation'

import { EUserCivility } from '@enum/user/civility.user.enum'
import { IUser } from '@type/user/user.type'

export const UserAddressSchema: Schema = new Schema({
  city: {
    type: String,
    trim: true
  },
  postalCode: {
    type: String,
    trim: true
  },
  street: {
    type: String,
    trim: true
  },
  country: {
    type: String,
    trim: true
  }
})

export const UserSchema: Schema = new Schema(
  {
    accounts: [
      {
        type: Schema.Types.ObjectId,
        ref: 'Account'
      }
    ],
    address: UserAddressSchema,
    civility: {
      type: EUserCivility,
      enum: EUserCivility
    },
    email: {
      type: String,
      lowercase: true,
      trim: true,
      unique: true,
      required: true
    },
    firstname: {
      type: String,
      trim: true
    },
    isEmailValidated: {
      type: Boolean,
      default: false
    },
    lastActiveAccount: {
      type: Schema.Types.ObjectId,
      ref: 'Account'
    },
    lastConnection: {
      type: Date
    },
    lastname: {
      type: String,
      trim: true
    },
    password: {
      type: String,
      trim: true
    },
    phone: {
      type: String,
      trim: true
    },
    username: {
      type: String,
      trim: true
    }
  },
  {
    timestamps: true
  }
)

UserSchema.plugin(beautifyUnique)

export const User: Model<IUser & Document> = model<IUser & Document>('User', UserSchema, 'user')

@guillaumejauffret,

I’ve been able to reproduce the deprecation warning on my end as well. However, looking at the code, it shouldn’t be an issue.

Looking at your previous message

  • If I use ('user', UserSchema, 'user') with a local database URL: I can see the user collection and its records, but with a production database URL: Forest detects the collection but says there is no record.

Looking at the generated code from the forest-cli (Not typescript), we do generate ('user', UserSchema, 'user') and not ('User', UserSchema, 'user'), which explains the DEPRECATION WARNING.

Would it be possible to have the response of the HTTP GET call on /user in the production context, where forest tell you there is no records?

hi @jeffladiray thank you for your research.

Here are the logs in production:

2021-12-07T09:39:47.418237+00:00 heroku[router]: at=info method=OPTIONS path="/forest/user/count?fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&fields%5BlastActiveAccount%5D=_id&searchExtended=0&timezone=Europe%2FParis" host=slaf-forest-admin.herokuapp.com request_id=86f4d9e7-695a-48da-a7a8-516471de587c fwd="13.37.162.137" dyno=web.1 connect=0ms service=1ms status=204 bytes=420 protocol=https
2021-12-07T09:39:47.311346+00:00 heroku[router]: at=info method=GET path="/forest/user?timezone=Europe%2FParis&fields%5BlastActiveAccount%5D=_id&fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&page%5Bnumber%5D=1&page%5Bsize%5D=15&searchExtended=0&sort=-_id" host=slaf-forest-admin.herokuapp.com request_id=c1feda10-740f-4d7b-bbd2-2e89e930a782 fwd="13.37.162.137" dyno=web.1 connect=0ms service=16ms status=200 bytes=329 protocol=https
2021-12-07T09:39:47.473807+00:00 heroku[router]: at=info method=GET path="/forest/user/count?fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&fields%5BlastActiveAccount%5D=_id&searchExtended=0&timezone=Europe%2FParis" host=slaf-forest-admin.herokuapp.com request_id=94841e9d-5480-4c2e-94f0-db5aa5b85b8d fwd="13.37.162.137" dyno=web.1 connect=0ms service=7ms status=200 bytes=329 protocol=https
2021-12-07T09:39:47.245123+00:00 heroku[router]: at=info method=OPTIONS path="/forest/user?timezone=Europe%2FParis&fields%5BlastActiveAccount%5D=_id&fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&page%5Bnumber%5D=1&page%5Bsize%5D=15&searchExtended=0&sort=-_id" host=slaf-forest-admin.herokuapp.com request_id=420b68e6-05e1-436e-821a-6a0d86685f29 fwd="13.37.162.137" dyno=web.1 connect=0ms service=2ms status=204 bytes=439 protocol=https
2021-12-07T09:39:47.245966+00:00 app[web.1]: OPTIONS /forest/user?timezone=Europe%2FParis&fields%5BlastActiveAccount%5D=_id&fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&page%5Bnumber%5D=1&page%5Bsize%5D=15&searchExtended=0&sort=-_id 204 0 - 0.178 ms
2021-12-07T09:39:47.309379+00:00 app[web.1]: GET /forest/user?timezone=Europe%2FParis&fields%5BlastActiveAccount%5D=_id&fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&page%5Bnumber%5D=1&page%5Bsize%5D=15&searchExtended=0&sort=-_id 200 11 - 15.667 ms
2021-12-07T09:39:47.419078+00:00 app[web.1]: OPTIONS /forest/user/count?fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&fields%5BlastActiveAccount%5D=_id&searchExtended=0&timezone=Europe%2FParis 204 0 - 0.156 ms
2021-12-07T09:39:47.471703+00:00 app[web.1]: GET /forest/user/count?fields%5Buser%5D=_id%2CcreatedAt%2Cemail%2CisEmailValidated%2ClastActiveAccount%2ClastConnection%2Cpassword%2CupdatedAt%2Cusername&fields%5BlastActiveAccount%5D=_id&searchExtended=0&timezone=Europe%2FParis 200 11 - 6.131 ms

This is this response call I’m looking for. It should be available in the Network tabs of your browser console.

sorry…misunderstood,

we get empty array:

the request following this one is:

and the count:

This is weird indeed. I’m assuming that you already checked your mongodb and it already have records.

I can’t spot any misconfiguration on my end associated with both of your environment on the mentioned project.

When you are using upper cased model name, could you confirm that the associated route URL also contains the upper cased model name?

On my end, switching from user to User in the mongoose model definition definitely causes issue of the associated route (/forest/user) is not updated as well (/forest/User)

yes of course I have records in DB.

When I use upper cased model name the collection are not called because I get this in the UI:

I think I’m starting to understand.

Did you originally create your project with the forest-cli command (Not in-app), then switched your project to typescript?

Yes I created my forest backend with the forest-cli. I have not switched it to Typescript yet.

It’s my application backend which is in Typescript.

Here is my User model in the forest backend:

module.exports = (mongoose, Mongoose) => {
  // This section contains the properties of your model, mapped to your collection's properties.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const schema = Mongoose.Schema({
    'accounts': { type: [Mongoose.Schema.Types.ObjectId], ref: 'account' },
    'lastConnection': Date,
    'username': String,
    'lastActiveAccount': { type: Mongoose.Schema.Types.ObjectId, ref: 'account' },
    'email': String,
    'isEmailValidated': Boolean,
    'password': String,
  }, {
    timestamps: true,
  });

  return mongoose.model('user', schema, 'user');
};

Well, maybe I didn’t get everything then.

Both your application and agent are running on the same nodejs process?

nop. two different processes