Optimization issue: Smart fields are loaded even they are hidden on the UI

Help me!
1

Feature(s) impacted

Optimize the loading time for my collection on Data.
following https://docs.forestadmin.com/user-guide/collections/performance

Observed behavior

when collection is loading on Data I see on my docker logs that the smart fields are loading (with console logs) even on the UI I set them hidden .

image
image2792Ă—240 65.6 KB

for example this smart field is loaded (I see the log on console)

image
image974Ă—968 115 KB

Expected behavior

when I hide smart field it should not be loaded.

PS:
The collection used is Contact which is a Smart Collection.

  • Project name: clevermate dashboard
  • Environment name: Production
  • Agent (forest package) name & version: “forest-express-sequelize”: “9.3.5”
  • Database type: postgres
2

Hello @Adel_de_Clevermate :wave:,

Thank you for your feedback, we are going to investigate on this. :pray: I’ll let you know as soon as I have any news on this subject.

Kind regards,

Florian

3

Can you give me the code that gets the records of your smart collection? :slight_smile:

4

Hi @Florian_Gonzales

Yes sure.
Here is code of get endpoint for the smart collection Contacts:

const attributes_ = {
  email: "user_email",
  phone: "phone_1",
  addresse_lat: "location_lat",
....
};
const contactCustomColumns = Object.keys(attributes_).map(
  (k) => `${attributes_[k]} as ${k}`
);

router.get("/contacts", (req, res, next) => {
  const limit = parseInt(req.query.page.size) || 20;
  const offset = (parseInt(req.query.page.number) - 1) * limit;
  const queryType = sequelize.QueryTypes.SELECT;
  let conditionSearch = "";

  if (req.query.search) {
    if (isNaN(req.query.search))
      conditionSearch = ` c_users.user_email LIKE '%${req.query.search.replace(
        /\'/g,
        "''"
      )}%' OR c_users.phone_1 LIKE '%${
        req.query.search
      }%' OR c_users.user_first_name LIKE '%${
        req.query.search
      }%' OR c_users.user_last_name LIKE '%${req.query.search}%' `;
    else
      conditionSearch = ` c_users.phone_1 LIKE '%${req.query.search}%' OR id= ${req.query.search}`;
  }

  const isContactCondition =
    "user_tutor=0 and ((customer_id is not null and customer_id <> '') or status='Client')"; //removed  and desinscrit_at is null
  const queryData = `
    SELECT *, ${contactCustomColumns}
    FROM c_users WHERE ${isContactCondition}
    ${conditionSearch ? `AND (${conditionSearch})` : ""}
    ORDER BY CASE WHEN c_users.due_date IS NOT NULL THEN 0 ELSE 1 END, c_users.due_date ASC, c_users.updated_at DESC
    LIMIT ${limit}
    OFFSET ${offset}
  `;


  const queryCount = `
    SELECT COUNT(*)
    FROM c_users WHERE ${isContactCondition}
      ${conditionSearch ? `AND ${conditionSearch}` : ""}
  `;

  Promise.all([
    sequelize.query(queryData, { type: queryType }),
    sequelize.query(queryCount, { type: queryType }),
  ])
    .then(async ([customerStatsList, customerStatsCount]) => {
      const customerStatsSerializer = new RecordSerializer({
        name: "contacts",
      });
      const customerStats = await customerStatsSerializer.serialize(
        customerStatsList
      );
      const count = customerStatsCount[0].count;
      res.send({ ...customerStats, meta: { count: count } });
    })
    .catch((err) => {
      console.log("error ==", err)
      next(err)
    }
      );
});
1 Like
5

I’ll try to reproduce your set-up with a smart collection and the endpoint code you’ve just sent me. :slight_smile: In a classic case of smart field we don’t have this problem. :thinking: Thanks for your feedback :pray:, I’ll keep you posted!

1 Like
6

Hello @Adel_de_Clevermate :wave:

When you want to serialise records, you also need to specify which fields you want to be serialised. On top of that, if a smart field has been required to be serialised, then the serialiser will be responsible of getting that smart field value.

In fact, for a smart collection, this is a missing feature. To workaround that, you will need to use the RecordsGetter instead. Please switch from RecordSerialiser to RecordsGetter, and serialise using it:

const recordsGetter = new RecordsGetter({ name: 'contacts' }, req.user, req.query);

const fieldsPerModel = Object.keys(req.query.fields).reduce((fields, modelName) => {
    fields[modelName] = req.query.fields[modelName].split(',');
    return fields;
}, {});
recordsGetter.fieldsPerModel = fieldsPerModel;
const customerStats = await recordsGetter.serialize(
    customerStatsList
);

This sounds hacky for sure, but that will do the trick.

Please tell me if that helps :slight_smile:

Steve.

2 Likes
7

For people that might be interested, we have just released a fix so you can get the behaviour you expect natively.

From forest-express-sequelize v9.3.8 and forest-express-mongoose v9.3.12, the RecordSerializer now accepts a new last argument, allowing you to specify the smart field you want to compute.

In your route code, you should be provided with the fields requested by the UI, you can get those fields and give them to your RecordsSerializer so the hidden smart fields will not be computed anymore:

router.get("/contacts", (req, res, next) => {

  const customerStatsSerializer = new RecordSerializer({
    name: "contacts",
  }, null, req.query);

  ...

  const customerStats = await customerStatsSerializer.serialize(
    customerStatsList
  );

  ...
})

I hope this can help someone :slight_smile:

Steve.

2 Likes