Links array of strings

[usr1vang]

Hello! Im my project im working with mongo database. There I have a Payments collection as follows:

        "payments" : [
                {
                        "_id" : ObjectId("614c7daa61490a8940fa550d"),
                        "amount" : 35,
                        "transactions" : [
                                "614c7daa61490a8940fa550b",
                                "614c7daa61490a0080fa550a",
                        ],
                }
        ],

Within the payments field is the transactions field that contains the transaction ids corresponding to the transaction collection. I would like to be able to link those ids and take me to the corresponding transaction but since this field is of type [‘string’] I can’t achieve the objective.

Is there any way to achieve the goal? I thought about maybe making a smart collection where I create objects each one with a transaction, that is to say, starting from the json data shown above obtain:

        "payments" : [
                {
                        "_id" : ObjectId("614c7daa61490a8940fa550d"),
                        "amount" : 35,
                        "transactions" : [
                                "614c7daa61490a8940fa550b",
                        ],
                }
        ],
        "payments" : [
                {
                        "_id" : ObjectId("614c7daa61490a8940fa550d"),
                        "amount" : 35,
                        "transactions" : [
                                "614c7daa61490a0080fa550a",
                        ],
                }
        ],

but I haven’t really been able to get it to work.
I appreciate any help!
Thank you!

[jeffladiray]

Hi @usr1vang

Did you take a look at either smart relationships and flatten nested fields in our mongoose documentation?

If either of theses does not suit your need, for sure smart collections will, but i’m pretty sure you should be able to achieve with goal with the first 2 proposed solutions.

Don’t hesitate if you have a bit of code to share, I’ll be happy to help.

[usr1vang]

Thanks for your reply!

I’m a little confused on how to achieve the goal, I think a smart relationship would work but I’m not sure how to implement it given my schematics.

In my project I have a model called portfolios and another one called transactions.
In the portfolios model, one of its fields is payments:

As you can see in the image inside the payments field one of its fields in transactions, which is an array of strings containing the ids of the transactions. (these ids correspond to the ids of each transaction in this model).

Since forest takes the payments field as JSON, in order to visualize it in a nice way let’s say, create a smart collection viewPayments:

Now what I want to achieve is to be able to link those ids inside the transactions field in my smart collection and be directed to the corresponding transaction with that id.

I may have to create a smart relationship inside the smart collection viewPayments but I’m not sure how to do that and where I should create the routes to achieve what I want.

I would appreciate if you could help me with some code.

Thanks!

[jeffladiray]

@usr1vang,

Would it be possible to have the model (.js) definition for both Portfolios, Charities, Payment & Transactions model (As text, so I can reproduce easily your setup on my end)?

I don’t get why, in your case, payments.transactions is not an array of ObjectId pointing directly to a transaction. With such configuration, flattening mongoose field should be enough to display it as a “Payment has many transactions”, which seems to be the behavior you want to have?

Thanks in advance.

[usr1vang]

You say define payments.trnsactions as:

 transactions: {
            type: [Mongoose.Schema.Types.ObjectId],
            ref: "transactions",
            required: true,
            default: [],
          },

?

[jeffladiray]

@user1vang,

If the transactions in your payment objects are actually ObjectId referencing transactions, you should indicate them as it is instead of String.

Our new flatten mechanism is really powerful for this kind of usage, so if you indicate in your code that payment has many transaction (via an array of ObjectId referencing transactions), the documentation I mentionned earlier about flatten nested fields should give you your expected result.

Let me know.

[usr1vang]

I have define my transaction object as you mentioned. My new portfolio schemas now is:

  const schema = Mongoose.Schema(
    {
      status: {
        type: String,
        required: true,
        default: "active",
      },
      name: {
        type: String,
        trim: true,
      },
 
  . . . . . .

      payments: [
           . . . 
          amount: {
            type: Number,
            required: true,
            min: [0, "Invalid number"],
          },
          . . .
          transactions: {
            type: [Mongoose.Schema.Types.ObjectId],
            ref: "transactions",
            required: true,
            default: [],
          },
        },
      ],

and define in forest->portfolios.js fieldsToFlatten: [“payments”], But im still view the following:

[jeffladiray]

I’m not a 100% sure, but I guess the transactions definition should defined as

      transactions: {
        type: [{ type: Mongoose.Schema.Types.ObjectId, ref: 'transactions' }],
        required: true,
        default: [],
      },

Did you try with the following syntax ?

(Documentation from mongoose).

[usr1vang]

Yes also, but there is no case.

I created a smart collection Payments to be able to visualize the payment portfolios. This is my code:

Is there a possibility to separate the payments by transaction in order to view each payment transaction separately? i.e:

{
  currency: 'usd',
  type: 'one_time',
  created_at: 2021-09-23T10:50:22.961Z,
  transactions: [ 614c7e761601ed6be3ebe1e8, 6152f8dcced1fb2b8823fd7c ],
  _id: 614c7e761601ed21fdebe1ea,
  amount: 35,
  start_date: 2021-09-23T13:17:42.658Z,
  end_date: 2021-09-23T13:17:42.658Z,
  payment_id: 614c7e761601ed21fdebe1ea,
  id: 61530c5e584e4432f7516e43
}
{
  currency: 'usd',
  type: 'one_time',
  created_at: 2021-09-23T10:50:22.961Z,
  transactions: [ 614c7e761601ed6be3ebe1e8, 6152f8dcced1fb2b8823fd7c ],
  _id: 615c7e761601ed21fdebe1ea,
  amount: 35,
  start_date: 2021-09-23T13:17:42.658Z,
  end_date: 2021-09-23T13:17:42.658Z,
  payment_id: 615c7e761601ed21fdebe1ea,
  id: 61530c5f584e4432f7516e44
}

to

{
 ...
  transactions: 614c7e761601ed6be3ebe1e8
  _id: 614c7e761601ed21fdebe1ea,
...
  payment_id: 614c7e761601ed21fdebe1ea,
  id: 61530c5e584e4432f7516e43
}, 
{
 ...
  transactions: 6152f8dcced1fb2b8823fd7c
  _id: 614c7e761601ed21fdebe1ea,
...
  payment_id: 614c7e761601ed21fdebe1ea,
  id: 61530c5e584e4432f7516e43
}
{
  ...
  transactions: 614c7e761601ed6be3ebe1e8,
  _id: 615c7e761601ed21fdebe1ea,
 ...
  payment_id: 615c7e761601ed21fdebe1ea,
  id: 61530c5f584e4432f7516e44
}
{
  ...
  transactions: 6152f8dcced1fb2b8823fd7c,
  _id: 615c7e761601ed21fdebe1ea,
 ...
  payment_id: 615c7e761601ed21fdebe1ea,
  id: 61530c5f584e4432f7516e44
}

If there is this possibility, what should I add to my code? because I have not been able to achieve it.

Thanks for your time!

[jeffladiray]

I just did a quick test on my end, which is giving me what I think is what you are expecting

If you setup differs from mine, just tell me what to change, but it appear to me that a simple fieldsToFlatten: ['payments'], on the portfolios model is enough to have the related data transactions.

Please let me know

[usr1vang]

That looks good. But how have you defined portfolio.payments?

because the same is a json array, i.e. payments: [ {},{},etc] and where each of those json has several fields among them transactions which is an array of ids (transactions: [{ type: Mongoose.Schema.Types.ObjectId, ref: “transactions” }]).

If I remove the square brackets I get the panel like you, but it doesn’t display the data: “There are no payment transaction yet”, which I guess is because it can’t read the data from the database, because the schema is defined differently from how the data is in the database (json array).

[jeffladiray]

I’ve shared on my previous message a screenshot of my data structure for portfolios, but it looks like

{
    "_id" : ObjectId("60d18c167e1a0fc070c99fce"),
    "name" : "Test1",
    "payments" : {
        "transactions" : [ 
            ObjectId("615318025ce85ecbd65d521d")
        ],
        "amount" : 10
    }
}

which looks similar to the one you shared previously (Except for a few fields I removed to ease the setup)

[usr1vang]

It is not working on my side. Below i attached my portfolio.payments

It not display the payments transactions:

And also when i want to create one transaction there its display the error:

Portfolios 614c7cf061490a8940fa54e9 update failed: Plan executor error during findAndModify
:: caused by :: Cannot create field ‘transactions’ in element
{payments: [ { _id: ObjectId(‘614c7daa61490a8940fa550d’), amount: 35,
currency: “usd”, type: “one_time”, created_at: new Date(1632402159010),
transactions: [ “614c7daa61490a8940fa550b” ], start_date: new Date(1632402858341),
end_date: new Date(1632402858341) } ]}

[jeffladiray]

Let restart from the beginning, since i’m actually not sure. Are your payments.transactions actual ObjectId or are they just string?

If they are just string, indeed this will not work, since they will not be linked to actual transactions, thus my initial smart relationship solution. (Especially this part which contains an example similar to what you are trying to achieve)

Still, if they are actual ObjectId referencing transactions, I would highly suggest to migrate them to their true type, then a simple fieldToFlatten: ['payements'] will work, as shown by my previous example.

Let me knbow if that helps

[usr1vang]

Yes, my payments.transactions are actual ObjectId referencing transactions.

I don’t understand what it means when you say “migrate them to their true type”.

Also i forgot to say than when i push ‘Create the first one’ in the Payments->transactions seccion and it display the error i detail in the previous post, the transaction in the transaction collection is created anyway, in spite of the displayed error.

[jeffladiray]

I’ve just code a smart relationship based on your definition. Could you check and tell me if this is the expected result ?

routes/portfolios.js

...

router.get('/portfolios/:recordId/relationships/transactions', permissionMiddlewareCreator.list(), async (req, res, next) => {
  const record = await portfolios.findById(req.params.recordId);
  if (record?.payments) {
    const transactionIds = record.payments.reduce((acc, payment) => {
      return acc.concat(payment.transactions);
    }, []);

    const transacs = await transactions.find({
      '_id': { $in: transactionIds }
    })
    const serializer = new Liana.RecordSerializer(portfolios);
    res.json(await serializer.serialize(transacs, { count: transacs.length }));
  }
});

...

forest/portfolios.js

const { collection } = require('forest-express-mongoose');

// This file allows you to add to your Forest UI:
// - Smart actions: https://docs.forestadmin.com/documentation/reference-guide/actions/create-and-manage-smart-actions
// - Smart fields: https://docs.forestadmin.com/documentation/reference-guide/fields/create-and-manage-smart-fields
// - Smart relationships: https://docs.forestadmin.com/documentation/reference-guide/relationships/create-a-smart-relationship
// - Smart segments: https://docs.forestadmin.com/documentation/reference-guide/segments/smart-segments
collection('portfolios', {
  actions: [],
  fields: [{
    field: 'transactions',
    type: ['String'],
    reference: 'transactions._id'
  }],
  segments: [],
  fieldsToFlatten: [],
});

[usr1vang]

No, that’s not what I’m looking for. Let’s start over, it may have gotten a little confusing.
I have a Portfolios collection whose scheme is:

As you can see, inside portfolios->payments is the transactions field, which is a string array corresponding to each id of the Transactions collection.

An example of portfolios->payments is:

Because forest admin does not have a “pretty” way to display portfolio->payments in the panel, i create a smart collection called Paymentes to display these fields.
My code for the smart collection Payments is:

This smart collection Payments mentioned in order to have a nice view of the portfolio->payments is displayed in the forest admin panel like this:

What I would like to achieve if it is possible, is to be able to visualize in the panel in the smart collection that I create Payments a row for each transaction, that is to say in the example of the previous picture, to see three rows:
Payments id 1 → transaction[0]
Payments id 2 → transaction[0]
Payments id 2 → transaction[1]

I think this should be possible, modifying the code of the router.get(
“/portfolios/:recordId/relationships/Payments” …
but the truth is that I have tried all the ways and I have not been able to get it.

I await your comments,
Thank you very much!

[jeffladiray]

forest

const { collection } = require('forest-express-mongoose');
collection('smartPayments', {
  actions: [],
  fields: [{
    field: 'id',
    type: 'String',
  }, {
    field: 'transactionId',
    type: 'String'
  },{
    field: 'name',
    type: 'String'
  }],
  segments: [],
  fieldsToFlatten: [],
});

route

const uuid = require('uuid');

router.get('/portfolios/:recordId/relationships/smartPayments', permissionMiddlewareCreator.list(), async (req, res, next) => {
  const record = await portfolios.findById(req.params.recordId);
  if (record?.payments) {
    let smartPayments = await record.payments.reduce((acc, payment) => {
      return Promise.all(payment.transactions.map(async (tranId) => {
        const tr = await transactions.findById(tranId);
        return { id: uuid.v4(), transactionId: tranId, name: tr.name };
      }).concat(acc));
    }, []);
    smartPayments = smartPayments.flat(); 

    const serializer = new Liana.RecordSerializer({ name: 'smartPayments' });
    res.json(await serializer.serialize(smartPayments, { count: smartPayments.length }));
  }
});

(The code is not optimize nor reviewed, I’m just sharing this as an example of how this could be implemented. Especially, it might not work as expected in your case ).

This should do the trick here. Note that you can add whatever fields you want as long as there both present in the route and in the model.
In that case, the result will show SmartPayments as a hasMany relationship, with one row per transaction.
Keep in mind that smart collections requires you to override all the routes (CRUD) in order to fully work as normal (mongoose) collection - Here i’m just showing that it should work as expected as a hasMany smart collection.

Let me know if that helps

[usr1vang]

YES! that’s what I needed, using part of that code I got what I was looking for. Thank you very much!!! I appreciate your attention and time!