Calculated fields and smart fields problems (regressions in 6.3.7 and 6.6.2?)

Help me!
11

Thank you @Guillaume_Cisco
I understand this is some kind of unusual problem, and I appreciate very much your effort to help me out. But I’m afraid I can’t tell what you are missing. I can only say that if you can’t reproduce then the setup you did is different from mine :man_shrugging:

Now I understood it can be helpful if I provide the most precise setup steps based upon lumber cli.
So I’ll try to do my best to give you any fine grained instruction to let you reproduce the issue with lumber cli. Follow me, let’s go! :+1:

Database

Create a Postgres database (i used testsq001 as the database name). Then run the following two scripts in your database sql console:

Tables
CREATE TABLE public.main (
  id integer NULL,
  amount numeric(5,2) NULL,
  CONSTRAINT newtable_pk PRIMARY KEY (id)
);

CREATE TABLE public.related (
  id integer NULL,
  id_main integer NULL,
  some_data varchar NULL,
  CONSTRAINT related_pk PRIMARY KEY (id),
  CONSTRAINT related_fk FOREIGN KEY (id_main) REFERENCES public.main(id) ON DELETE CASCADE ON UPDATE CASCADE
);
Data
INSERT INTO public.main VALUES (1, 1.5);
INSERT INTO public.related VALUES (1, 1, 'one');
INSERT INTO public.related VALUES (2, 1, 'two');

Forest

I created a new forest project (in my setup I named it test-sq-001) and I followed the directions from the forest project setup page:

Screenshot 2021-01-06 at 11.57.54
Screenshot 2021-01-06 at 11.57.541384×1410 85.9 KB

Now we have the project created from the lumber cli. In the project we have two model files: models/main.js and models/related.js.

Edit the file models/main.js (final resulting file below):

  • change the data type in the amount field (lumber cli maps sql numeric(5,2) to DOUBLE, but we need to use the DECIMAL(5,2)sequelize datatype) and
  • insert the (previously described) getter (with a useful console.log in it)

Here is the resulting models/main.js file:

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
module.exports = (sequelize, DataTypes) => {
  const { Sequelize } = sequelize;
  // This section contains the fields of your model, mapped to your table's columns.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const Main = sequelize.define('main', {
    amount: {
      // =============================================================
      // I commented out the following line
      // type: DataTypes.DOUBLE,
      //
      // and I added the following lines:
      type: DataTypes.DECIMAL(5,2),
      get() {
        console.log('amount', this.dataValues.amount ? this.dataValues.amount : this.dataValues)
        // Workaround until sequelize issue #8019 is fixed
        const value = this.dataValues.amount
        return value === null ? value : parseFloat(value)
      },
      // =============================================================
    },
  }, {
    tableName: 'main',
    timestamps: false,
    schema: process.env.DATABASE_SCHEMA,
  });

  // This section contains the relationships for this model. See: https://docs.forestadmin.com/documentation/v/v6/reference-guide/relationships#adding-relationships.
  Main.associate = (models) => {
    Main.hasMany(models.related, {
      foreignKey: {
        name: 'idMainKey',
        field: 'id_main',
      },
      as: 'idMainRelateds',
    });
  };

  return Main;
};

Nothing else was changed in the lumber generated project.

Let’s run npm run and you should be able to see the problem!!!

Evidence

This is a video showing what you are supposed to see:

And this is the (relevant) log:

amount 1.50
amount 1.50
amount 1.50
amount 1.50
GET /forest/main/1 304 - - 5.744 ms
amount { id: 1 }
amount { id: 1 }
amount { id: 1 }
amount { id: 1 }
GET /forest/main/1/relationships/idMainRelateds?fields%5Brelated%5D=id%2CidMain%2CsomeData&fields%5BidMain%5D=id&page%5Bnumber%5D=1&page%5Bsize%5D=15&timezone=Europe%2FRome&sort=-id 304 - - 9.386 ms
GET /forest/main/1/relationships/idMainRelateds/count?fields%5Brelated%5D=id%2CidMain%2CsomeData&fields%5BidMain%5D=id&timezone=Europe%2FRome 304 - - 6.219 ms

The console.log was hit 8 times while loading the summary page for the main record:

  • in the first 4 times (GET /forest/main/1) the this.dataValues.amount was correctly retrieved (1.5)
  • in the second 4 times (due to thee call GET /forest/main/1/relationships/idMainRelateds?.....) the this.dataValues.amount was undefined and the dataValues object was shown (no amount field there).

(why four + four times??? :thinking: can’t tell. But maybe you could explain?)

Possible workaround (and evidence of problem understanding)

Now, if I use the workaround described here below, everything works fine.
This is the workaround getter:

      get() {
        console.log('amount', this.dataValues.amount ? this.dataValues.amount : this.dataValues)
        // Workaround until sequelize issue #8019 is fixed
        const value = this.dataValues.amount
        return value === undefined || value === null ? value : parseFloat(value)
      },

With this workaround we let pass the undefined value as is (please, se my previous post for a better explanation).

Exclude this is due to the datatype

I eventually tried also restoring the DOUBLE sequelize datatype in the lumber generated model. This is the default “translation” by lumber cli.

I did this test to collect evidence that this problem is not directly due to the adoption of the DECIMAL sequelize datatype.
And nothing changes: we have this problem with the DOUBLE datatype too.

Conclusions and questions

Did you reproduce? :pray:

Questions:

  • May you please acknowledge my test and my explanations?
  • May you please better explain why this happens?
  • Don’t you think that the main record should not be re-loaded with the related data table?
  • Why the getter is called multiple (4) times?

Versions doublecheck

This is the package.json created by lumber cli:

{
  "name": "test-sq-001",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node ./server.js"
  },
  "dependencies": {
    "body-parser": "1.19.0",
    "chalk": "~1.1.3",
    "cookie-parser": "1.4.4",
    "cors": "2.8.5",
    "debug": "~4.0.1",
    "dotenv": "~6.1.0",
    "express": "~4.17.1",
    "express-jwt": "5.3.1",
    "forest-express-sequelize": "^6.0.0",
    "morgan": "1.9.1",
    "require-all": "^3.0.0",
    "sequelize": "~5.15.1",
    "pg": "~8.2.2"
  }
}

More versions on my side:

$ node -v && npm -v
v14.9.0
6.14.8

Thank you in advance.