Migration to `@forestadmin/agent` - when a createOverride is called an updateOverride is called immediately after

[Matteo]

We’re migrating from forest-express-sequelize to @forestadmin/agent .

We saw that when creating a record with a relationship, an update is called immediately after.

We have this collection BoatContentTranslation which belongs to a BoatContent collection. For both collections we have an override on both create and update.

When we create a boat-content-translation record the createOverride is called, passing the data from the input form (from the context we get this).

this override function returns this object

Anyways, after the create has correctly terminated, the updateOverride is immediately called (always from the boat-content-translation), passing this data (from the context we get this)

Which looks weird, because it is trying to update the idBoatContent, which is already present in the record and with the same value.

The problem is that (on our side) we use these overrides to fill in a “relational audit log”, which we need to track all changes to these collections (since it’s really important for us for a number of internal reasons…) and since both the create and the update overrides have been called we get two log entries with almost the same timestamp (difference of just a few milliseconds).

And inspecting the logs we see that they are “grouped” two by two (first create then update) and the only difference between each “couple”'s before and after is the updated_at.

The same also happens if we try to create a boat-content-translation directly from its list view (not starting from the boat-content).

We’d like to know if there’s something we are missing in the return value of the createOverride that is causing this issue or if this is due to something else. Because we can’t understand why the updateOverride gets called after every createOverride. But in our perspective it shouldn’t.

So please let us know what we’re probably missing

Here is the “meta” section of the forest admin schema file:

  {
    "liana": "forest-nodejs-agent",
    "liana_version": "1.41.7",
    "liana_features": null,
    "stack": {"engine": "nodejs", "engine_version": "20.11.1"}
  }

and from package.json:

"sequelize": "^6.28.0"

cc @julien.jolles

Thank you,
Matteo

[vince]

Hi @Matteo,

I’m investigating but for the moment I can’t reproduce. I’ll let you know as soon as I find something

[vince]

I’ve found an issue that might be producing what you are describing. I’ll let you know once the fix is released

[vince]

Hey @Matteo,

I released a potential fix. COuld you let me know if it works on your side ?

[Matteo]

Thank you @vince

I’m afraid nothing changed on our side
The problem is still there.

Matteo

[vince]

COuld you share a video reproducing your issue ? You can send it in private

[Matteo]

@vince I just sent you a private message with the video.
Please confirm that you received it

Thank you.
Matteo

[vince]

Yes I received it

Looking at it I see no issue on the app. COuld you show me how you are creating your boat logs ?

[Matteo]

@vince

Looking at it I see no issue on the app.

What do you mean? I think I’m misunderstanding

As you can see in the video, there is a createOverride and one updateOverride immediately after.

Isn’t this an issue?
Is this the expected behavior on your side?

May you please elaborate?

Thank you,
Matteo

[vince]

Short answer: yes, it’s the expected behaviour.

To elaborate a bit more:

When you create a new Has Many / BelongsTo you are first creating the record and then update its relationship.
So if you have users has many comments called userComments, when you create a new comment in the userComments relationship

• We create first the comment
• Then we notify the agent the userComments has been updated with the new comment

Why ? Because it actually depends on your database / implementations.
For example, in mongoDB database you would have an array field on the user table that is a list of id of comments.

So in your case, you are certainly considering the update even when there is no change ?

[Matteo]

Thank you @vince

Of course, we’re considering the update because the updateOverride function gets called on the related collection.

We don’t doublecheck if you’re issueing a unecessary update
And this specific relation update is unecessary on our side, because we don’t have MongoDB. We have Postgresql, which is a relational DB and the foreign key sits in the “related” table, which have been already created with the parent primary key specified.

In a relational DB perspective, you are just doubling all the DB queries with an additional unecessary update query.

But we deeper dug into this problem on our side. And we can confirm that the http calls from forest to our backend didn’t change.

…So, why we didn’t have this problem before?
Follow me along to find the answer, please

You (from online forest) send two http calls:

• che creation to the route POST /forest/BoatContentTranslation
• the update to the relationship to the route POST /forest/BoatContent/111/relationships/BoatContentTranslations

We used to attach all our audit logs routines to the following http calls (with specific route overrides):

• create: /BoatContentTranslation
• update: /BoatContentTranslation/:idBoatContentTranslation

But with the new agent, you are piping all the update http calls into just the one single function updateOverride! Here is the point.

So the final question is:
how do we distingish, with the new agent, the two kinds on interaction (update vs. relation update) from forest?

At the moment, as a workaround, we tried to prevent the unecessary update with a hook on the collection. Here is the code:

contents.addHook('Before', 'Update', (ctx) => { 
    if ('idBoatContent' in ctx.patch && isIdBoatContentUnchanged(ctx))
      ctx.throwForbiddenError('Pretend nothing happened (unnecessary update performed by forest have been prevented)')
  }) 

But this is the result:

That’s not very nice-looking (in a user perspective), actually…

And we need an additional query on our side in the isIdBoatContentUnchanged function. Which is required to get the previous data and state if something relevant changed or not…

More over, we would need to apply this workaround to all the collections where this problem arises…

So this is not a feasible solution on our end.

It would be very nice to have something in the context that allows to know the reason for the update (to match the route which the update comes from, in the previous implementation on our end).

Let me know your thoughts, please

cc @jeffladiray @julien.jolles

Thank you,
Matteo

[Enki]

Hello @Matteo ,

Dissociating relation update vs record update is not really possible since a record update from the frontend can also trigger a relation update if only the relation is updated.
A @vince said, dissociating creation and relation update is a technical requirement for us to handle any sort of DB.

A solution for your issue would be to rely on activity logs instead of the update hook. But it would certainly need some work on your side to adapt your system.

[Matteo]

Thank you @Enki

a record update from the frontend can also trigger a relation update if only the relation is updated

I think you mean that it may trigger… if you have a DB like MongoDB.
But as we already said, we have Postgres, and it’s not the case.

So this is not impossible. This is the price for your generalization.

This is why I suggested the following:

It would be very nice to have something in the context that allows to know the reason for the update (to match the route which the update comes from, in the previous implementation on our end).

Let me know your thoughts, please.
Matteo

[Enki]

@Matteo

I think you mean that it may trigger… if you have a DB like MongoDB.

I was saying that if you want to update the field “idBoatContent” of your record, it will trigger a relation update instead of a regular record update, because the field is used as a foreign key.
The query to update a relationship field in a record (real update situation) is the same query that is done after creating such a record.
So even if you ignored the relationship updates, you would potentially ignore meaningfull updates treated as relationship updates.
The fact that the frontend sends an update query after the creation is indeed caused by the frontend being agnostic of the datasources and agent type, and this is not modifiable.

What you are suggesting is more a feature request than a bug fix. And as an other solution exist to handle the case of getting data on collection updates (activity logs), it may never be prioritised.

kind regards,
Enki

[Matteo]

@Enki you’re right.
This is a feature request, which is needed because of the lack of such a feature in the new agent version. I’m just trying to be proactive. And this is supposed to be appreciated.

Thank you for a feedback.
Matteo

PS: activity logs is not a solution

1. because my audit logs features do not match your features
2. because the problem here is to hav information on my side about why the override function is being called, among the various sources (as it happened before, with the old agent version)