Missing relation btw records in Many 2 many tables

We migrated our data from airtable to postgres. When I access the postgres DB directly , the DB is fine, all the tables and relations btw tables are fine.
But in forestadmin dashboard, for two specific tables, I’m not able to see data from table B that are linked to a specific record in table A. The link btw these two tables is 2 many to many relationships.

These are the models :

type ProcessOutline struct {
	Model

	Name        string `gorm:"column:name" json:"Process"`
	Description string `gorm:"column:description" json:"Description,omitempty"`
	Status      string `gorm:"column:status" json:"Template Status,omitempty"`

	InitiativeOutline   *InitiativeOutline
	InitiativeOutlineID uint           `gorm:"column:initiative_outline_id" json:"Associated Initiative,omitempty"`
	ChildTaskOutlines   []*TaskOutline `gorm:"many2many:process_outlines_child_task_outlines;association_jointable_foreignkey:child_task_outline_id" json:"Child Tasks,omitempty"`
	ParentTaskOutlines  []*TaskOutline `gorm:"many2many:process_outlines_parent_task_outlines;association_jointable_foreignkey:parent_task_outline_id" json:"Triggered by (Task),omitempty"`

	PrerequisiteProcesses []*ProcessOutline `gorm:"many2many:pre_requisite_process_outlines;association_jointable_foreignkey:pre_requisite_process_outline_id" json:"Prerequisites (prior processes),omitempty"`
}
type TaskOutline struct {
	Model

	Name                  string `gorm:"column:name" json:"Task"`
	Description           string `gorm:"column:description" json:"Description,omitempty"`
	TaskType              string `gorm:"column:type" json:"Task type,omitempty"`
	Status                string `gorm:"column:status" json:"Template done?,omitempty"`
	OwnerType             string `gorm:"column:owner_type" json:"Owner Type,omitempty"`
	Timing                string `gorm:"column:timing" json:"Timing,omitempty"`
	ParameterDisplayNames string `gorm:"column:parameter_display_names" json:"Parameter Display Names,omitempty"`

	RequiredKeyParameters []*ParameterOutline `gorm:"many2many:task_outlines_parameter_outlines" json:"Required Key Parameters,omitempty"`
	AssociatedDocuments   []*DocumentOutline  `gorm:"many2many:task_outlines_document_outlines" json:"Associated Document,omitempty"`
	ParentProcesses       []*ProcessOutline   `gorm:"many2many:process_outlines_child_task_outlines;jointable_foreignkey:child_task_outline_id" json:"Parent Process,omitempty"`
	TriggeredProcesses    []*ProcessOutline   `gorm:"many2many:process_outlines_parent_task_outlines;jointable_foreignkey:parent_task_outline_id" json:"Triggered Process,omitempty"`

	PrerequisiteTasks []*TaskOutline `gorm:"many2many:pre_requisite_task_outlines;association_jointable_foreignkey:pre_requisite_task_outline_id" json:"Predecessor / blocker tasks,omitempty"`
}

I suspect that forestadmin is getting confused about the double many to many relationship.

Any ideas of what can be the cause or what where I can investigate further?

Expected behavior

We should see data linked btw two tables (many2many)

Actual behavior

When I click on a specific record on table A that should be linked to other records from table B I don’t see anything.

Failure Logs

Please include any relevant log snippets, if necessary.

Context

Please provide any relevant information about your setup.

  • Package Version:
  • Express Version: 4.16.4
  • Sequelize Version: 5.15.2
  • Forest-express: 7.3.1
  • Forest-express-sequelize:6.3.2
  • Database Dialect: postgres
  • Database Version:12.3.2
  • Project Name:
1 Like

Hi @lowly2005, nice to meet you and welcome on Forest!

I do not recognize these models format, is it a kind of C declaration?

Anyway, can you please share the DDL from your Postgres database (only relevant tables)? SQL instructions recreating your tables and their relationships will allow me to reproduce your situation.

Thank you

Hello Sliman,

It is in Golang.

This is the DDL :

SQL instructions to generate these tables :

-- public.process_outlines definition

-- Drop table

-- DROP TABLE public.process_outlines;

CREATE TABLE public.process_outlines (
	id serial NOT NULL,
	created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	deleted_at timestamp NULL,
	"name" varchar(100) NOT NULL,
	status varchar(50) NULL,
	description varchar(2000) NULL,
	initiative_outline_id int4 NULL,
	CONSTRAINT process_outlines_name_key UNIQUE (name),
	CONSTRAINT process_outlines_pkey PRIMARY KEY (id)
);

-- public.task_outlines definition

-- Drop table

-- DROP TABLE public.task_outlines;

CREATE TABLE public.task_outlines (
	id serial NOT NULL,
	created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	deleted_at timestamp NULL,
	"name" varchar(100) NOT NULL,
	description varchar(2000) NULL,
	status varchar(50) NULL,
	"type" varchar(50) NULL,
	owner_type varchar(50) NULL,
	timing varchar(50) NULL,
	parameter_display_names varchar(1000) NULL,
	initiative_outline_id int4 NULL,
	CONSTRAINT task_outlines_name_key UNIQUE (name),
	CONSTRAINT task_outlines_pkey PRIMARY KEY (id)
);
-- public.pre_requisite_process_outlines definition

-- Drop table

-- DROP TABLE public.pre_requisite_process_outlines;

CREATE TABLE public.pre_requisite_process_outlines (
	process_outline_id int4 NOT NULL,
	pre_requisite_process_outline_id int4 NOT NULL,
	CONSTRAINT pre_requisite_process_outlines_pkey PRIMARY KEY (process_outline_id, pre_requisite_process_outline_id)
);


-- public.pre_requisite_process_outlines foreign keys

ALTER TABLE public.pre_requisite_process_outlines ADD CONSTRAINT pre_requisite_process_outline_pre_requisite_process_outlin_fkey FOREIGN KEY (pre_requisite_process_outline_id) REFERENCES process_outlines(id) ON UPDATE CASCADE;
ALTER TABLE public.pre_requisite_process_outlines ADD CONSTRAINT pre_requisite_process_outlines_process_outline_id_fkey FOREIGN KEY (process_outline_id) REFERENCES process_outlines(id) ON UPDATE CASCADE;
-- public.process_outlines_child_task_outlines definition

-- Drop table

-- DROP TABLE public.process_outlines_child_task_outlines;

CREATE TABLE public.process_outlines_child_task_outlines (
	process_outline_id int4 NOT NULL,
	child_task_outline_id int4 NOT NULL,
	CONSTRAINT process_outlines_child_task_outlines_pkey PRIMARY KEY (process_outline_id, child_task_outline_id)
);


-- public.process_outlines_child_task_outlines foreign keys

ALTER TABLE public.process_outlines_child_task_outlines ADD CONSTRAINT process_outlines_child_task_outlines_child_task_outline_id_fkey FOREIGN KEY (child_task_outline_id) REFERENCES task_outlines(id) ON UPDATE CASCADE;
ALTER TABLE public.process_outlines_child_task_outlines ADD CONSTRAINT process_outlines_child_task_outlines_process_outline_id_fkey FOREIGN KEY (process_outline_id) REFERENCES process_outlines(id) ON UPDATE CASCADE;

Thanks

Hi @lowly2005,

Thanks for all the details you provided.
It would also help a lot if you could share the models generated by the lumber CLI during your onboarding. Especially the model definitions for the tables process_outlines and task_outlines.
The odds are high that it is a problem of relationship definition in those models.

Thanks for your help.

Hello @arnaud,

These are the models:

Process_outlines

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
module.exports = (sequelize, DataTypes) => {
  const { Sequelize } = sequelize;
  // This section contains the fields of your model, mapped to your table's columns.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const ProcessOutlines = sequelize.define('processOutlines', {
    createdAt: {
      type: DataTypes.DATE,
      defaultValue: Sequelize.literal('CURRENT_TIMESTAMP'),
    },
    updatedAt: {
      type: DataTypes.DATE,
      defaultValue: Sequelize.literal('CURRENT_TIMESTAMP'),
    },
    deletedAt: {
      type: DataTypes.DATE,
    },
    name: {
      type: DataTypes.STRING,
    },
    status: {
      type: DataTypes.STRING,
    },
    description: {
      type: DataTypes.STRING,
    },
  }, {
    tableName: 'process_outlines',
    underscored: true,
    schema: process.env.DATABASE_SCHEMA,
  });

  // This section contains the relationships for this model. See: https://docs.forestadmin.com/documentation/v/v6/reference-guide/relationships#adding-relationships.
  ProcessOutlines.associate = (models) => {
    ProcessOutlines.belongsTo(models.initiativeOutlines, {
      foreignKey: {
        name: 'initiativeOutlineIdKey',
        field: 'initiative_outline_id',
      },
      as: 'initiativeOutline',
    });
    // NB:BK, Being someone's PreRequisite doesn't make this process a PostRequisite,
    // but we'll use this term for lack of a better word.
    ProcessOutlines.belongsToMany(models.processOutlines, {
      through: 'preRequisiteProcessOutlines',
      foreignKey: 'pre_requisite_process_outline_id',
      otherKey: 'process_outline_id',
      as: 'PostRequisiteProcessOutlines',
    });
    ProcessOutlines.belongsToMany(models.processOutlines, {
      through: 'preRequisiteProcessOutlines',
      foreignKey: 'process_outline_id',
      otherKey: 'pre_requisite_process_outline_id',
      as: 'PreRequisiteProcessOutlines',
    });
    ProcessOutlines.belongsToMany(models.taskOutlines, {
      through: 'processOutlinesChildTaskOutlines',
      foreignKey: 'process_outline_id',
      otherKey: 'child_task_outline_id',
    });
    ProcessOutlines.belongsToMany(models.taskOutlines, {
      through: 'processOutlinesParentTaskOutlines',
      foreignKey: 'process_outline_id',
      otherKey: 'parent_task_outline_id',
    });
    ProcessOutlines.hasMany(models.processData, {
      foreignKey: {
        name: 'processOutlineIdKey',
        field: 'process_outline_id',
      },
      as: 'processOutlineProcessData',
    });
  };

  return ProcessOutlines;
};

Task_outlines

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
module.exports = (sequelize, DataTypes) => {
  const { Sequelize } = sequelize;
  // This section contains the fields of your model, mapped to your table's columns.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const TaskOutlines = sequelize.define('taskOutlines', {
    createdAt: {
      type: DataTypes.DATE,
      defaultValue: Sequelize.literal('CURRENT_TIMESTAMP'),
    },
    updatedAt: {
      type: DataTypes.DATE,
      defaultValue: Sequelize.literal('CURRENT_TIMESTAMP'),
    },
    deletedAt: {
      type: DataTypes.DATE,
    },
    name: {
      type: DataTypes.STRING,
    },
    description: {
      type: DataTypes.STRING,
    },
    status: {
      type: DataTypes.STRING,
    },
    type: {
      type: DataTypes.STRING,
    },
    ownerType: {
      type: DataTypes.STRING,
    },
    timing: {
      type: DataTypes.STRING,
    },
    parameterDisplayNames: {
      type: DataTypes.STRING,
    },
  }, {
    tableName: 'task_outlines',
    underscored: true,
    schema: process.env.DATABASE_SCHEMA,
  });

  // This section contains the relationships for this model. See: https://docs.forestadmin.com/documentation/v/v6/reference-guide/relationships#adding-relationships.
  TaskOutlines.associate = (models) => {
    TaskOutlines.belongsTo(models.initiativeOutlines, {
      foreignKey: {
        name: 'initiativeOutlineIdKey',
        field: 'initiative_outline_id',
      },
      as: 'initiativeOutline',
    });
    TaskOutlines.belongsToMany(models.processOutlines, {
      through: 'processOutlinesChildTaskOutlines',
      foreignKey: 'child_task_outline_id',
      otherKey: 'process_outline_id',
    });
    TaskOutlines.belongsToMany(models.processOutlines, {
      through: 'processOutlinesParentTaskOutlines',
      foreignKey: 'parent_task_outline_id',
      otherKey: 'process_outline_id',
    });
    TaskOutlines.belongsToMany(models.parameterOutlines, {
      through: 'taskOutlinesParameterOutlines',
      foreignKey: 'task_outline_id',
      otherKey: 'parameter_outline_id',
    });
    TaskOutlines.belongsToMany(models.documentOutlines, {
      through: 'taskOutlinesDocumentOutlines',
      foreignKey: 'task_outline_id',
      otherKey: 'document_outline_id',
    });
    // NB:BK, Being someone's PreRequisite doesn't make this task a PostRequisite,
    // but we'll use this term for lack of a better word.
    TaskOutlines.belongsToMany(models.taskOutlines, {
      through: 'preRequisiteTaskOutlines',
      foreignKey: 'pre_requisite_task_outline_id',
      otherKey: 'task_outline_id',
      as: 'PostRequisiteTaskOutlines',
    });
    TaskOutlines.belongsToMany(models.taskOutlines, {
      through: 'preRequisiteTaskOutlines',
      foreignKey: 'task_outline_id',
      otherKey: 'pre_requisite_task_outline_id',
      as: 'PreRequisiteTaskOutlines',
    });
    TaskOutlines.hasMany(models.taskData, {
      foreignKey: {
        name: 'taskOutlineIdKey',
        field: 'task_outline_id',
      },
      as: 'taskOutlineTaskData',
    });
  };

  return TaskOutlines;
};

Process_outlines_Child_task_outlines

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
module.exports = (sequelize, DataTypes) => {
  const { Sequelize } = sequelize;
  // This section contains the fields of your model, mapped to your table's columns.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const ProcessOutlinesChildTaskOutlines = sequelize.define('processOutlinesChildTaskOutlines', {
    processOutlineId: {
      type: DataTypes.INTEGER,
      primaryKey: true,
    },
    childTaskOutlineId: {
      type: DataTypes.INTEGER,
      primaryKey: true,
    },
  }, {
    tableName: 'process_outlines_child_task_outlines',
    underscored: true,
    timestamps: false,
    schema: process.env.DATABASE_SCHEMA,
  });

  // This section contains the relationships for this model. See: https://docs.forestadmin.com/documentation/v/v6/reference-guide/relationships#adding-relationships.
  ProcessOutlinesChildTaskOutlines.associate = (models) => {
    ProcessOutlinesChildTaskOutlines.belongsTo(models.taskOutlines, {
      foreignKey: {
        name: 'childTaskOutlineIdKey',
        field: 'child_task_outline_id',
      },
      as: 'childTaskOutline',
    });
    ProcessOutlinesChildTaskOutlines.belongsTo(models.processOutlines, {
      foreignKey: {
        name: 'processOutlineIdKey',
        field: 'process_outline_id',
      },
      as: 'processOutline',
    });
  };

  return ProcessOutlinesChildTaskOutlines;
};

Process_outlines_Parent_task_outlines

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
module.exports = (sequelize, DataTypes) => {
  const { Sequelize } = sequelize;
  // This section contains the fields of your model, mapped to your table's columns.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const ProcessOutlinesParentTaskOutlines = sequelize.define('processOutlinesParentTaskOutlines', {
    processOutlineId: {
      type: DataTypes.INTEGER,
      primaryKey: true,
    },
    parentTaskOutlineId: {
      type: DataTypes.INTEGER,
      primaryKey: true,
    },
  }, {
    tableName: 'process_outlines_parent_task_outlines',
    underscored: true,
    timestamps: false,
    schema: process.env.DATABASE_SCHEMA,
  });

  // This section contains the relationships for this model. See: https://docs.forestadmin.com/documentation/v/v6/reference-guide/relationships#adding-relationships.
  ProcessOutlinesParentTaskOutlines.associate = (models) => {
    ProcessOutlinesParentTaskOutlines.belongsTo(models.taskOutlines, {
      foreignKey: {
        name: 'parentTaskOutlineIdKey',
        field: 'parent_task_outline_id',
      },
      as: 'parentTaskOutline',
    });
    ProcessOutlinesParentTaskOutlines.belongsTo(models.processOutlines, {
      foreignKey: {
        name: 'processOutlineIdKey',
        field: 'process_outline_id',
      },
      as: 'processOutline',
    });
  };

  return ProcessOutlinesParentTaskOutlines;
};

I hope this helps.

thanks

Thanks.

As it looks like you have multiple belongsToMany on models.taskOutlines, I wonder how it can work properly without aliases (ie as attribute) to distinguish both relationships on a similar model.

Could you try the code below in ProcessOutlines model to see if it fixes your issue?

ProcessOutlines.belongsToMany(models.taskOutlines, {
  through: 'processOutlinesChildTaskOutlines',
  foreignKey: 'process_outline_id',
  otherKey: 'child_task_outline_id',
+ as: 'aliasOfFirstRelationship' // NOTICE: use a business oriented alias
});
 ProcessOutlines.belongsToMany(models.taskOutlines, {
  through: 'processOutlinesParentTaskOutlines',
  foreignKey: 'process_outline_id',
  otherKey: 'parent_task_outline_id',
+ as: 'aliasOfSecondRelationship' // NOTICE: use a business oriented alias
});

And try the same in TaskOutlines models?

TaskOutlines.belongsToMany(models.processOutlines, {
  through: 'processOutlinesChildTaskOutlines',
  foreignKey: 'child_task_outline_id',
  otherKey: 'process_outline_id',
+ as: 'aliasOfFirstRelationship' // NOTICE: use a business oriented alias
});
TaskOutlines.belongsToMany(models.processOutlines, {
  through: 'processOutlinesParentTaskOutlines',
  foreignKey: 'parent_task_outline_id',
  otherKey: 'process_outline_id',
+ as: 'aliasOfSecondRelationship' // NOTICE: use a business oriented alias
});

@arnaud what do you think about auto-generating as with a value equal to through? It’ll fix the case in this post.

One might argue that it still leaves edge case for the following model, but that anyway breaks without as and needs to be handled as a special case on top.

    // NB: BK, Being someone's PreRequisite doesn't make this process a PostRequisite,
    // but we'll use this term for lack of a better word.
    ProcessOutlines.belongsToMany(models.processOutlines, {
      through: 'preRequisiteProcessOutlines',
      foreignKey: 'pre_requisite_process_outline_id',
      otherKey: 'process_outline_id',
      as: 'PostRequisiteProcessOutlines',
    });
    ProcessOutlines.belongsToMany(models.processOutlines, {
      through: 'preRequisiteProcessOutlines',
      foreignKey: 'process_outline_id',
      otherKey: 'pre_requisite_process_outline_id',
      as: 'PreRequisiteProcessOutlines',
    });

Stay safe, stay healthy!
-BK

2 Likes

Hi @BK42,

I’ll go even further, the best idea to prevent any edge cases is probably to setup an alias to each models relationships (belongsTo, hasMany, belongsToMany) even if in 95% of the cases it would not be a really necessary configuration (ie no identical relationship on the same targeted model).

through is probably a good base (after the targeted model) for belongsToMany relationships aliases.
As you mentioned, the final model generation would need a final check to validate that relationships aliases are unique in each models, this also something we could support in the future.

1 Like