Failing: reading 'foreignCollection'

David_Roegiers · August 23, 2023, 7:01am

Feature(s) impacted

I’m calling the following function somewhere, and this leads to an error according to the failure logs below. The relationships addressed in the second argument of the list function are configured correctly as many-to-one relationships, since I can validate it in the UI. Any idea what might be the issue here?

...
const items = await context.dataSource.getCollection('legalProductOrders_items').list(
        {
          conditionTree: {
            field: 'parentId',
            operator: 'In',
            value: records.map((r) => r._id.toString()),
          },
        },
        ['legalProductItem:legalProduct:nameEN', 'parentId']
      );
...

Failure Logs

===== An exception was raised =====
GET /forest/legalProductOrders?{
 timezone: Europe/Zurich,
 fields[client__manyToOne]: _id,
 fields[legalProductOrders]: _id,client__manyToOne,cogs,completionDate,createdAt,fulfillmentStatus,notionPageId,paymentStatus,product,serviceProviderInvoiceReceived,serviceProviderOrderAccepted,source,serviceProviderOrderSubmitted,updatedAt,sourceMedium,
 page[number]: 1,
 page[size]: 15,
 sort: -createdAt
}

 Cannot read properties of undefined (reading 'foreignCollection') 

TypeError: Cannot read properties of undefined (reading 'foreignCollection')
    at rewriteField (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/helpers/rewrite-projection.ts:10:68)
    at /Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/collection.ts:58:68
    at Projection.map (<anonymous>)
    at Projection.replace (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-toolkit/src/interfaces/query/projection/index.ts:29:17)
    at ComputedCollection.list (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/collection.ts:58:40)
    at RelaxedCollection.list (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/context/relaxed-wrappers/collection.ts:120:34)
    at Object.getValues (/Users/davidroegiers/Jurata/jurata-next/forest-2/customizations/legal-product-orders.ts:54:88)
    at /Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/helpers/compute-fields.ts:23:56
    at transformUniqueValues (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/utils/deduplication.ts:26:31)
    at computeField (/Users/davidroegiers/Jurata/jurata-next/forest-2/node_modules/@forestadmin/datasource-customizer/src/decorators/computed/helpers/compute-fields.ts:15:31)
===================================

Context

Project name: jurata
Team name: jurata-2
Environment name: Development
Agent (forest package) name & version: "@forestadmin/agent": "^1.13.1"
Database type: MongoDB
Recent changes made on your end if any: green field solution

David_Roegiers · August 23, 2023, 7:09am

I found one item in my DB, where the relationship to legalProductItem could not be established (wrong entry). I imagine that is the reason behind this error.

Assuming that is true, it’s unfortunate, that a single false record in the DB, causes the entire call, and therefore the UI, to fail. How would you work around this? In other words, how would you make such a call work, efficiently, while just defaulting to undefined or an empty string for “broken” records.

Alban_Bertolini · August 23, 2023, 8:59am

Hello,
Appreciate your input. I’ve managed to replicate the issue by including an incorrect relation name in the projection list.
My assumption is that either legalProductItem or legalProduct is not accurate on your end. If you’re employing TypeScript, your IDE should flag an error within your projection list. Perhaps a spelling or capitalization error?

The error message in the log isn’t very informative; enhancing it on our end is a possibility.

David_Roegiers · August 23, 2023, 12:40pm

Hi,

Thanks for getting back to me.

There is no Typescript error, according to the IDE, nor a visible typo for me.

Alban_Bertolini · August 23, 2023, 1:43pm

Could you put a break point in rewrite-projection.js at the line 10 and display collection.schema.fields to check if the prefix match one of the fields properties ?

David_Roegiers · August 24, 2023, 2:37pm

Hello @Alban_Bertolini @jeffladiray

Sorry, for the late reply. I had to focus on something else.

It seems the relationship is not present in the schema on that line - not sure why.

As requested:

Alban_Bertolini · August 24, 2023, 2:53pm

Is it possible to have more details about how you build the relations between legalProductOrders_items and legalProductItem? Maybe you use the flattener. It will probably help me to reproduce the bug.

David_Roegiers · August 25, 2023, 1:02pm

I will share a repo what’s your Github name?

David_Roegiers · August 25, 2023, 1:05pm

This is most likely you, so I added this account to the repo.

https://github.com/access2justice/legal-product-item-forest

David_Roegiers · August 28, 2023, 11:25am

Hi @Alban_Bertolini, is there any progress on this?

Alban_Bertolini · August 28, 2023, 11:28am

Sorry, My github account is scra3: Scra3 (scra) · GitHub.

David_Roegiers · August 28, 2023, 11:41am

Thanks, I just added you.

Alban_Bertolini · August 30, 2023, 8:29am

After a pm discussion. There is a bug.

To understand why there is a bug

When you are using the addField, the customization relationship can be computed before or after the execution of the addField because sometimes we need to have the field to compute a relation and sometimes we need the relation to compute the added field .

Example

.customizeCollection('aCollection', collection => {
  // compute a many to one between on aCollection.
  aCollection.addManyToOneRelation('aManyToOneAddedByAddManyToOneRelation', ....);
})
.customizeCollection('MyCollection', collection => {
   // the field that uses a calculated relationship to be computed
     collection.addField('aField', {
        columnType: 'String',
        dependencies: ['id'],
        getValues: async (records, context) => {
         // the collection with the computed many to one.
          const items = await context.dataSource.getCollection('aCollection').list(
            {
              conditionTree: {
                field: 'id',
                operator: 'In',
                value: records.map(r => r.id.toString()),
              },
            },
            // the many to one writen above.
            ['aManyToOneAddedByAddManyToOneRelation:id'], // <= This line is the source of the error.
          );

          return items.map(item => ...);
        },
      });
    })

Currently, the code is throwing this error: TypeError: Cannot read properties of undefined (reading ‘foreignCollection’) because we compute the field BEFORE the relation aManyToOneAddedByAddManyToOneRelation and the relation does not exist at this time. The forest agent checks only the dependencies fields to know if the field must be computed before or after the computed relation. It is a Bug.

Workaround

A little trick is to add an existing relationship to the dependencies field to force the agent to compute the field before the computed relationship.

Example

.customizeCollection('MyCollection', collection => {
      collection.addField('aField', {
        columnType: 'String',
        dependencies: ['id', 'myRelation:id'], // <= This line is the tip.

Action

We have created a bug ticket for this bug.

David_Roegiers · August 30, 2023, 10:09am

Thanks for the summary, Alban!

Any information about when this bug would be fixed?

Or can you imagine a workaround, so we can move forward? Maybe inverting the relationship between aCollection and MyCollection, so that all the required data can be called using the dependencies field of addField? Not sure how to achieve that though …

Alban_Bertolini · August 30, 2023, 10:21am

Please try to add the client__manyToOne:_id relation field in the dependencies properties in your addField method. It will force the field to be computed after the relation.

If you have an error again, is the error always the same? If it is, you have probably a cycle dependency.
A field needs a computed relation to be computed and the relation needs this same field to be computed .

Alban_Bertolini · September 1, 2023, 8:22am

David_Roegiers Are you still blocked?

David_Roegiers · September 1, 2023, 12:52pm

@Alban_Bertolini

Thanks for the suggestion. I didn’t have a chance to try it out yet, will do a.s.a.p. What do you mean by “Are you already blocked?”?

Alban_Bertolini · September 1, 2023, 1:26pm

I wanted to know if the workaround works with your problem.

David_Roegiers · September 6, 2023, 1:01pm

Hi Alban, sorry for my late reply.

No, unfortunately, this does not solve the problem. I don’t know how it would, since the client-relationship has nothing to do with it.

I do get the same error, always the same Cannot read properties of undefined (reading 'foreignCollection')

There are no cyclic dependencies.

Alban_Bertolini · September 6, 2023, 1:50pm

Ty for the feedback.
I’ll get to work on fixing the bug as soon as possible, and then, if the problem persists on your side, we’ll do a pair programming session.