Pagination not working (forest-express-mongoose)

Feature(s) impacted

Pagination. All collections.

Observed behavior

All our collections display the same 10 results, no matter what page, or how we adjust page size. This is only an issue in our hosted staging and production environments. Pagination works normally in local development.

Expected behavior

Working pagination in our hosted environments.

Failure Logs

I overrode one of our list routes in order to log the request params. Here’s what was logged for req.params in local dev vs staging.

Local dev

{
  timezone: 'America/Los_Angeles',
  fields: {
    Creator: 'name,email,_id,acceptedUserAgreement,assetFiles,assetLinks,bio,contentCategory,coverPhotoId,createdAt,description,location,password,profilePhotoId,stage,stripeConnectId,stripeCustomerId,updatedAt,userType,valuation,username,fundingRaised'
  },
  page: { number: '2', size: '10' },
  sort: '-_id'
}

Staging

{
  'page[size]': '10',
  sort: '-_id',
  'page[number]': '2',
  timezone: 'America/Los_Angeles',
  'fields[Creator]': 'name,bio,email,_id,acceptedUserAgreement,assetFiles,assetLinks,contentCategory,coverPhotoId,createdAt,description,location,password,profilePhotoId,stage,stripeConnectId,stripeCustomerId,updatedAt,userType,valuation,username,fundingRaised'
}

Seems like we might be encountering a shallow parsing issue? Question is still why the difference between environments, local vs remote?

Context

  • Project name: subscribe.so
  • Team name: AE Studio
  • Environment name: Staging, Production
  • Agent type & version: forest-express-mongoose@7.8.3

Our admin is hosted on vercel (serverless function).

1 Like

Hello @MasonLS and welcome on our community forum :wave:

In order to help you, I’d have a few more questions to ask you:

  • Are your local environment and production one under the same forest-express-mongoose version ?
  • Do you remember when it first appear ?
  • As long as your logs are different on local environment and production one: Did you do some recent changes on your code ?

Kind regards,
Louis

Hello @lclisson and @MasonLS :wave:

Thanks @MasonLS for the console.log you provided :raised_hands:

I just would like to mention few things to give some more context on this issue.

After a debug session we performed with @MasonLS we identified that the pagination was working well on development, however things started to break on remote environments (eg staging and prod).

At the first look, the mongoose queries constructed to perform the pagination on remote environments did not correspond to the pagination sent from the UI. No matter which page and how many records per page you ask, queries were always retrieving the first 10 records (eg page 0 and records per page 10).

I realised this is the default behaviour of the forest-express-mongoose agent in case no pagination options are provided, or incorrectly parsed.

My bet would be that somewhere and somehow the agent is not able to retrieve the pagination options. As you mentioned @MasonLS , the logs clearly show that the pagination options are indeed available and correct. We need to identify where the pagination options are incorrectly gathered.

Knowing that no other customers complained about such an issue, and that I could not reproduce on my end, I would definitely recommend to test our forest-express-mongoose integration on Vercel.

That’s it for me :smiley:

Steve.

OK I think I’ve identified the issue. And it is specific to Vercel.

Apparently Vercel tries to be helpful by automatically parsing incoming function requests. See these:

https://vercel.com/blog/vercel-node-helpers
https://vercel.com/docs/runtimes

Per the end of that latter doc, you can disable these helpers by setting NODEJS_HELPERS=0. Did this in staging and now pagination is fixed :partying_face:

So I guess our express app (or the forest client?) sees the request params are already parsed and doesn’t bother. And Vercel’s helper doesn’t handle nested query params.

Anyways, thanks for the help guys. This was a tough one :hot_face:

3 Likes

Hello @MasonLS :wave:

That’s a great news and I’m happy to see that you tackled this one :tada:

Indeed that was a tough one :exploding_head:

Thanks for filling the thread in with your solution.

Steve.