Problem with unattended behavior of the serialization process

Help me!
1

Hi team,

This issue is linked to the performance issue #1008

Looking for a way to override the default route that gets a list of users, I have found the following code in the doc, and worked from this base:

    const params = request.query;
    params.searchExtended = '1';

    const recordsGetter = new RecordsGetter(models.users);
    const users = await recordsGetter.getAll(params);
    const usersSerialized = await recordsGetter.serialize(users);

    return response.send(usersSerialized);

My need was to add a specific condition when getting the users.

1. First try
I replaced the call to getAll function by my own construction to get the records.
And, at the end, I called:

const usersSerialized = await recordsGetter.serialize(users);

Problem: When calling the serialize function, all the smart fields of my associated collections are computed, and are generating performance issues (see #1008)

I looked at the code, and I detected that the fieldsPerModel property of class recordsGetter is initialized in getAll method. Because I cannot call the getAll method, I am getting this described issue.

2. Second try
Then, I tried this code

const recordSerializer = new RecordSerializer({ name: 'users' });
const usersSerialized = await recordSerializer.serialize(users)

Problem: it reproduced the same issue with useless smart fields computations.

Question
Once the users records are built, what is the right way to serialize them, without generating the smart fields computation issue?
Do you have another solution?
Or would you able to provide a fix to make my proposal(s) work?

Thanks in advance for your help.

My setup:

    "database_type": "postgres",
    "liana": "forest-express-sequelize",
    "liana_version": "6.3.13",
    "engine": "nodejs",
    "engine_version": "12.13.1",
    "framework": "express",
    "framework_version": "^4.17.1",
    "orm_version": "4.44.4"
2

Hi @Louis-Marie,

Can you please describe the business case that leads you to add a specific condition on users listing?
I imagine you need to always work with a subset of the users table, for example?

I ask this question because, depending on your needs, I may redirect you into full-supported features of ForestAdmin like scopes. This would be a safer+stronger solution for you.

Thank you

3

Hi @Sliman_Medini,

Of course, here is our business case. It is a very simple need.

Our users need to easily find some customer using their “firstname lastname” or vice versa.
We are starting to have a large customer database, and for common names, it is not sufficient to enter either the first name, or the last name.
And it is much easier to enter ““firstname lastname”” in the search bar, rather to use the filters.

As an example: our users can type “asu camer”, and our code will find all the following customers:

  • Jaminasu Cameron
  • Cameron Asus
    …

The staff team who is using Forest admin all the day, needs very efficient ways of doing their recurrent tasks, and that one is a quick trick that does the job!

Best

4

Hello @Louis-Marie,

I carefully read your issue and I’d like to know if this solution could fit your need.
I assume you have declared a smart field entitled like “fullname”.

Could you add a search method to this smart field as described here?
Something like:

get: (user) => {
      return user.firstname + ' ' + user.lastname;
    },
search: function (query, search) {
      let split = search.split(' ');

      const searchCondition = {
        [Op.or]: [
          { [Op.and]: [
              { firstname: { [Op.like]: `%${split[0]}%` } },
              { lastname: { [Op.like]: `%${split[1]}%` } },
            ] },
          { [Op.and]: [
              { firstname: { [Op.like]: `%${split[1]}%` } },
              { lastname: { [Op.like]: `%${split[0]}%` } },
            ] },
        ]
      };

      query.where[Op.and][0][Op.or].push(searchCondition);

      return query;
    }

The get method is always called after the sql query aka the search method.
This way, only the records returned by the db will have their smart fields computed.

Would that work?

5

Hi @Guillaume_Cisco

Thank you for your proposal.
Indeed, previously, this was exactly the way in which we had declared the fullname (this was a smart field and the same mechanism for the search).

It was working, yes.

However, because of performance problems linked to the use of smart fields (performance issue and performance issue #1008), I have decided to move multiple smart fields definitions to native SQL fields.

As you can see, I already made many many different tries in order to solve all my issues.
The things improve little by little, but I would prefer to keep my current setup.

Your solution would be a good workaround if there were no other problem (and restrictions) with the use of smart fields, but this is not the case yet.

And we should definitively have a way to serialize the data without generating other issues. I think that it could be fixed quickly indeed.

Thanks for your help.

6

Thank you @Louis-Marie, I understand the complexity of your situation.
Currently the suggested solution is the way to go.
But you can still rewrite the recordsGetter.getAll and recordsGetter.serialize for answering your needs.

I will still push to the product board what you are facing for us to come with a better solution.