Duplicate Record with NULL foreign key error

SoftwareBridgementKe · October 18, 2021, 7:49pm

Expected behavior

I am trying to Duplicate a row through Forest Admin using the default functionality, and using NULL for a value for a foreign key (by leaving that field blank). This Foreign key allows null through the DB and NULL should be allowed as an input.

Actual behavior

[forest] 🌳🌳🌳 Unexpected error: related DbRefinance with pk null does not exist.

When duplicating a record and using a NULL value, an error is thrown because it is trying to look for the value of NULL in the DbRefinance table which is linked to the row that is being duplicated.

Is there a way to indicate to forest that the value is allowed NULL and to not search for a relation?

Failure Logs

[forest] 🌳🌳🌳 Unexpected error: related DbRefinance with pk null does not exist.

The error seems to be originating from the association-record.js file in the forest-express-sequelize package.

"use strict";

var _interopRequireDefault = require("@babel/runtime/helpers/interopRequireDefault");

require("core-js/modules/es.promise.js");

var _orm = _interopRequireDefault(require("./orm"));

async function get(model, pk) {
  const record = await _orm.default.findRecord(model, pk);

  if (!record) {
    console.log(model)
    throw new Error(`related ${model.name} with pk ${pk} does not exist.`);
  }

  return record;
}

exports.get = get;

Context

Please provide any relevant information about your setup.

Package Version: “forest-express-sequelize”: “^8.3.2”,
Express Version: “express”: “^4.17.1”,
Sequelize Version: “sequelize”: “6.6.2”,
Database Dialect: postgresql

anon34731316 · October 20, 2021, 3:24pm

Hello @SoftwareBridgementKe,

Thanks for your message and sorry for the delay.

I just tried on my side, but could not reproduce the issue you are talking about.

When simply creating a record (not duplicating), is it working well when leaving the foreign key blank field?
Could you please share your model definition (from your admin backend)?
If a get a chance to do a little screencast of what you’re facing, it could be great to be sure I perfectly understood your case.

Thanks!

SoftwareBridgementKe · October 20, 2021, 5:05pm

Yes, strangely on a normal creation of a record leaving the fields blank does work.

Below is the model we are using. The key in question is the RefinanceId which is the foreign key to the DbRefinance table. This issue also appears for CollectionId. Both are allowed null.

import {
    AllowNull,
    AutoIncrement,
    BelongsTo,
    Column,
    CreatedAt,
    DataType,
    Default,
    ForeignKey,
    Model,
    PrimaryKey,
    Table
} from 'sequelize-typescript';

import EnumHelpers from '../../Common/Helpers/EnumHelpers';
import { TransactionTypes } from '../../Core/Domain/Enums/TransactionTypes';
import { DbCollection } from './DbCollection';
import { DbJuristicEntity } from './DbJuristicEntity';
import { DbFinanceAgreement } from './DbFinanceAgreement';
import { DbRefinance } from './DbRefinance';

@Table({ tableName: 'AccountTransactions', updatedAt: false })
export class DbAccountTransaction extends Model {
    @PrimaryKey
    @AutoIncrement
    @Column(DataType.INTEGER)
    Id: number;

    @CreatedAt
    @Default(new Date())
    @Column(DataType.DATE)
    CreatedOn: Date;

    @ForeignKey(() => DbJuristicEntity)
    @Column(DataType.INTEGER)
    JuristicEntityId: number;

    @BelongsTo(() => DbJuristicEntity)
    JuristicEntity: DbJuristicEntity;

    @ForeignKey(() => DbFinanceAgreement)
    @Column(DataType.INTEGER)
    FinanceAgreementId: number;

    @BelongsTo(() => DbFinanceAgreement)
    FinanceAgreement: DbFinanceAgreement;

    @Column(DataType.ENUM(...EnumHelpers.GetNames(TransactionTypes)))
    TransactionType: TransactionTypes;

    @AllowNull(true)
    @Column(DataType.INTEGER)
    InstallmentNumber?: number;

    @Column(DataType.FLOAT)
    Amount: number;

    @AllowNull(true)
    @Column(DataType.STRING)
    BookkeepingTransactionId: string;

    @Column(DataType.DATE)
    TransactionDate: Date;

    @AllowNull(true)
    @Column(DataType.STRING)
    Description: string;

    @AllowNull(true)
    @Column(DataType.STRING)
    InternalNotes: string;

    @ForeignKey(() => DbCollection)
    @AllowNull(true)
    @Column(DataType.INTEGER)
    CollectionId?: number;

    @BelongsTo(() => DbCollection)
    Collection?: DbCollection;

    @AllowNull(true)
    @ForeignKey(() => DbRefinance)
    @Column(DataType.INTEGER)
    RefinanceId?: number;

    @BelongsTo(() => DbRefinance)
    Refinance?: DbRefinance;
}

I will try get a chance to get a screen capture of the issue. To explain it a bit more, it is essentially the same as creating a new record, I have tested with the exact same values, but for some reason that error is only thrown on the duplicate action.

anon34731316 · October 21, 2021, 8:38am

@SoftwareBridgementKe,

Thank you for your answer!.

Ok that’s very strange because the create and the duplicate record action should call the same route
Could you please share the request payload of both call (simple creation and duplication)?
That would be very insightful to understand what’s going on here!

Thanks again!

SoftwareBridgementKe · October 21, 2021, 9:07am

Here is the request payload for creating a new record:

//create record paylod
{
  "data": {
    "attributes": {
      "Amount": 0,
      "CreatedOn": "2021-10-12T09:00:34.000Z",
      "TransactionDate": "2021-10-22T09:01:29.000Z",
      "TransactionType": "AdvanceFeeVat"
    },
    "relationships": {
      "JuristicEntity": {
        "data": {
          "type": "DbJuristicEntities",
          "id": "6046"
        }
      },
      "FinanceAgreement": {
        "data": {
          "type": "DbFinanceAgreements",
          "id": "10895"
        }
      }
    },
    "type": "DbAccountTransactions"
  }
}

Here is the request payload for duplicating a record:

//duplicate record payload
{
  "data": {
    "attributes": {
      "Amount": 0,
      "CreatedOn": "2021-10-12T16:57:06.000Z",
      "Description": null,
      "InstallmentNumber": null,
      "InternalNotes": null,
      "TransactionDate": "2021-10-22T16:56:58.000Z",
      "TransactionType": "AdvanceFeeVat",
      "BookkeepingTransactionId": null
    },
    "relationships": {
      "Collection": {
        "data": null
      },
      "JuristicEntity": {
        "data": {
          "type": "DbJuristicEntities",
          "id": "6046"
        }
      },
      "Refinance": {
        "data": null
      },
      "FinanceAgreement": {
        "data": {
          "type": "DbFinanceAgreements",
          "id": "10895"
        }
      }
    },
    "type": "DbAccountTransactions"
  }
}

Seems like all the null fields are undefined in the creation payload.

anon34731316 · October 26, 2021, 8:15am

Hey @SoftwareBridgementKe,

Sorry for the delay.

I observe the same payload on my side but I just can’t reproduce the issue you’re facing.

Could you tell me what exact version of forest-express-sequelize you’re using?

And I’m not using Typescript on my side, but I am just curious: if you add a @AllowNull(true) just above the Refinance belongsTo definition (and same for the Collection), does it change anything to the problem?

Thanks.

SoftwareBridgementKe · October 26, 2021, 9:18am

I am currently testing on v8.4.4, but the issue was occuring on older versions as well.

Unfortunately the @AllowNull annotation is not allowed on properties that are not columns (big error is thrown on build). But the Refinance is defined as possibly being undefined.

jeffladiray · October 26, 2021, 3:28pm

Hi @SoftwareBridgementKe

Would it be possible to have the complete stacktrace of the issue you are encountering? Maybe running your agent with DEBUG=* might give further hint on the issue.

On my end, same as @anon34731316, I’m having the exact same behavior (null FK sent in the case of the duplicate), but no error …

Any further error logs would help

Thanks in advance

SoftwareBridgementKe · October 26, 2021, 3:40pm

This is the full stack trace of the error. Hope it’s helpful.

"stack": "Error: related DbCollection with pk null does not exist.
    at Object.get (/app/node_modules/forest-express-sequelize/dist/utils/association-record.js:13:11)
    at ResourceCreator._getTargetKey (/app/node_modules/forest-express-sequelize/dist/services/resource-creator.js:47:22)
    at ResourceCreator._makePromisesBeforeSave (/app/node_modules/forest-express-sequelize/dist/services/resource-creator.js:57:25)
From previous event:
    at Function.__NR_wrappedCast [as all] (/app/node_modules/newrelic/lib/shim/promise-shim.js:216:27)
    at ResourceCreator._handleSave (/app/node_modules/forest-express-sequelize/dist/services/resource-creator.js:96:15)
    at ResourceCreator.perform (/app/node_modules/forest-express-sequelize/dist/services/resource-creator.js:106:16)
    at /app/node_modules/forest-express/dist/routes/resources.js:85:91
    at PromiseShim.applySegment (/app/node_modules/newrelic/lib/shim/shim.js:1428:20)
    at __NR_wrappedThenHandler (/app/node_modules/newrelic/lib/shim/promise-shim.js:424:22)
    at /app/node_modules/sqreen/lib/instrumentation/hooks/clsAH.js:268:27
    at /app/node_modules/sqreen/lib/instrumentation/hooks/ns.js:29:27
    at bound (domain.js:413:15)
    at runBound (domain.js:424:12)
    at tryCatcher (/app/node_modules/forest-express/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:729:18)
    at Promise._fulfill (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:673:18)
    at Promise._resolveCallback (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:466:57)
    at Promise._settlePromiseFromHandler (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:559:17)
    at Promise._settlePromise (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:729:18)
    at Promise._fulfill (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:673:18)
    at PromiseArray._resolve (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:127:19)
    at PromiseArray._promiseFulfilled (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:145:14)
    at PromiseArray._iterate (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:115:31)
    at PromiseArray.init [as _init] (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:79:10)
    at bound (domain.js:413:15)
    at PromiseArray.runBound (domain.js:424:12)
    at Promise._settlePromise (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:601:21)
    at Promise._settlePromise0 (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:729:18)
    at Promise._fulfill (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise.js:673:18)
    at PromiseArray._resolve (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:127:19)
    at PromiseArray._promiseFulfilled (/app/node_modules/forest-express/node_modules/bluebird/js/release/promise_array.js:145:14)"

jeffladiray · October 27, 2021, 7:01am

@SoftwareBridgementKe, it’s definitely helpful.
Could you also share the definition of the model for either DbCollection or DbRefinance?

I’m still unable to reproduce, however I’ve got a better understanding of what’s going on.

SoftwareBridgementKe · October 27, 2021, 7:14am

Here is the DbRefinance Model:

import {
    BelongsTo,
    Column,
    DataType,
    ForeignKey,
    Model,
    PrimaryKey,
    Table,
    CreatedAt,
    Default,
    AutoIncrement,
    HasMany
} from 'sequelize-typescript';
import { DbFinanceAgreement } from './DbFinanceAgreement';
import { DbAccountTransaction } from './DbAccountTransaction';

@Table({ tableName: 'Refinances', updatedAt: false })
export class DbRefinance extends Model {
    @PrimaryKey
    @AutoIncrement
    @Column(DataType.INTEGER)
    Id: number;

    @CreatedAt
    @Default(new Date())
    @Column(DataType.DATE)
    CreatedOn: Date;

    @Column(DataType.FLOAT)
    Discount: number;

    @ForeignKey(() => DbFinanceAgreement)
    @Column(DataType.INTEGER)
    RefinancedFinanceAgreementId: number;

    @BelongsTo(() => DbFinanceAgreement, 'RefinancedFinanceAgreementId')
    RefinancedFinanceAgreement: DbFinanceAgreement;

    @ForeignKey(() => DbFinanceAgreement)
    @Column(DataType.INTEGER)
    NewFinanceAgreementId: number;

    @BelongsTo(() => DbFinanceAgreement, 'NewFinanceAgreementId')
    NewFinanceAgreement: DbFinanceAgreement;

    @Column(DataType.FLOAT)
    AmountBroughtForward: number;

    @Column(DataType.FLOAT)
    CollectionPortion: number;

    @HasMany(() => DbAccountTransaction)
    AccountTransactions: DbAccountTransaction[];
}

jeffladiray · October 27, 2021, 9:33am

Nice, thanks

I’m able to reproduce the issue & it seems like it is caused by your PK called Id instead of id.

This is a bug on our end so I’ll open a ticket. Be sure we’ll let you know once it fixes.

Thanks for your help

SoftwareBridgementKe · October 27, 2021, 9:49am

Ah, right I see. Yeah the column names in our db are a bit wonky.

Thanks a bunch.

SoftwareBridgementKe · November 8, 2021, 3:48pm

Hi there @jeffladiray,

Is there any progress on this issue?

jeffladiray · November 8, 2021, 3:53pm

Hey @SoftwareBridgementKe

The issue has been dispatched to the team in charge of the topic. You can follow the progression of the ticket here, but currently, no fix has been released.

Be sure we will let you know once the fix is released

SoftwareBridgementKe · December 3, 2021, 7:18am

Hi @jeffladiray,

Is this fix still being worked on? Has been quite a while now.

jeffladiray · December 3, 2021, 1:44pm

Hey @SoftwareBridgementKe

It seems like the ticket was dispatched to a pretty busy team. I moved it to another team, so the fix should hopefully be started shortly.

Sorry for the inconvenience.

JoshBridgement · January 24, 2022, 2:36pm

Hi @jeffladiray

I see that the ticket has been “waiting for review” for a while. Would it be possible to get a progress update?

Thanks a lot

Arnaud_Moncel · January 31, 2022, 11:07am

Hey guys @JoshBridgement @SoftwareBridgementKe
The ForestAdmin team apologizes for the long delay in resolving this issue. Thank you all for your patience.
Please upgrade forest-express-sequelize to 8.4.10 and let me know if that fix your issue.
Thank you again.

JoshBridgement · February 1, 2022, 7:05am

Thanks @Arnaud_Moncel. That seems to have fixed it

Topic		Replies	Views
Records created with literal "null" strings in some fields instead of database NULL Help me!	2	426	May 28, 2020
Can't save NOT NULL fields with empty strings Help me!	4	2456	October 6, 2020
Can't update a relation of a record with null value Help me!	3	271	January 27, 2021
Cannot create row from Forest Admin because identifier cannot be null Help me!	5	486	October 11, 2021
Cannot update the Smart Relationship (FK) of a record (or even insert it) Help me!	9	203	November 9, 2023

Duplicate Record with NULL foreign key error

Expected behavior

Actual behavior

Failure Logs

Context

Related Topics