Today I bumped up @forestadmin/datasource-mongoose (1.7.3 → 1.10.1) and mongoose (6.10.3 → 8.10.0). You may find the resulting package.json just below. I didn’t change anything else.
Can you give more information about the errors and when they occur ? is it directly at agent startup or when you display a specific collection ?
Is there any other change you did beside bumping up mongoose datasource and mongoose ?
If the purpose of your update was to benefit of the latest feature of @forestadmin/datasource-mongoose, I would suggest you to downgrade mongoose to a previous one to see if the performance is better.
Thanks for your feedback. The mere presence of issues on other repositories seems as a weak foundation for picking a direction of troubleshooting.
I went ahead and took an incremental approach to version bumping and interestingly:
I left everything the same, yet upgrading @forestadmin/mongoose from 1.7.6 to 1.7.7 introduce the problem and it does not get solved as I upgrade this to a higher version.
The issue is slightly different than before: the heap out of memory does not take place. It is merely the Detail view of single record, which simply does not load without any feedback.
Can you please help me with some pointers, what new functionality in that version bump has been altered? Or how can I debug?
I understand your point, to be able to effectively troubleshoot in an incremental way, are you able to consistently reproduce the initial issue you observed on 1.10.1 ?
I think it might have come as us “shooting the gun” with this performance issue despite the vast list between 1.7.3 and 1.10.1 but there is actually not that much changes in @forestadmin/datasource-mongoose between those versions, this is due to how our mono-repo cascades the releases of any packages and its dependants. You can find a somewhat readable list of changelogs here (I agree it’s not really that great to understand what’s going on).
I left everything the same, yet upgrading @forestadmin/mongoose from 1.7.6 to 1.7.7 introduce the problem and it does not get solved as I upgrade this to a higher version.
Anyway to come back to the topic, the version you mentioned 1.7.7, does not stem from @forestadmin/datasource-mongoose changes, thus my inquiry about reproducible steps to observe the performance issue.
Can you please help me with some pointers, what new functionality in that version bump has been altered?
You can refer back to the changeLog if any version seems suspicious to you but IMO notable versions to check would be:
1.7.6
1.7.8
1.7.12
1.8.1
Last point would be that @forestadmin/datasource-mongoose is compatible with Mongoose v8 starting from v1.9.0. Mongoose v8 models are not compatible with the introspection code before that.
Or how can I debug?
If you’re able to detect a version causing issues, you could then enable the debug mode on Mongoose, and analyse the aggregation pipeline that is generated, we would be glad to look at it if you’re able to find a problematic one.
“@forestadmin/datasource-mongoose”: “1.7.6” → everything works smoothly
“@forestadmin/datasource-mongoose”: “1.7.7” + no other changes → load feature stalls
So, the issue will uncovered in datasource-toolkit … I’m digging deeper. I’ll ask again: Can you share an overview of the changes in that version release. I’m a bit lost in your repo with the versioning kraken.
In the same manner, you can track the version of @forestadmin/datasource-toolkit by the changelog file here and it’s associated PR.
Which version of Mongoose are you using in parallel of “@forestadmin/datasource-mongoose”: “1.7.6” and “@forestadmin/datasource-mongoose”: “1.7.6” that results into stallings ?
I have tried checking everything and don’t get anywhere with the troubleshooting. Also checked the PR of datasource-toolkit 33 → 34. We are blocked by this issue and are stuck with a legacy database version and that is really problematic for us.
This single-thread back and forth will not get us anywhere. It’s been 18 days. Can we set up a call to get into the issue, please?
One more clue: this is happening on every collection, no matter how much or how little functionality is hosted on that collection: when I want to open a Detail view of a record, the system simply stalls and the UI maintains the loading state without timeout.
The issue stems from the incompatible versions that were installed as dependencies between @forestadmin/agent and @forestadmin/datasource-mongoose.
After bumping node to 18 and pulling the latest on @forestadmin/agent and @forestadmin/datasource-mongoose, we confirmed that the agent works as expected.
I have noted that it might not be clear (even though it wasn’t your case here), but the procedure we advice is to bump every @forestadmin packages to its latest version at the same time, to ensure compatibility. This strategy, however, presents flaws, as it would not be possible to tell easily which version of an earlier package is compatible with another. I’ll share this feedback with the team, maybe we can do something better on this.
