Making smart fields searchable (filterable)

Help me!
1

Feature(s) impacted

Smart fields (first_name, last_name) and their search/filter functionality.

Observed behavior

I have defined two smart fields, first_name and last_name, which retrieve data from another table using a complex query. When I use collection.emulateFieldFiltering('first_name'), it works but is very slow. When I attempt to manually override IContains using collection.replaceFieldOperator, I get an error stating:
Operator replacement cycle: profilesUser.first_name[IContains] -> profilesUser.first_name[IContains]

Expected behavior

I expect the first_name and last_name smart fields to be searchable and filterable without significant performance issues or errors.

Failure Logs

Here is the relevant code snippet:

const {fetchSubmissionItemValue} = require("../../utilities/reusable/fetchSubmissionItemValue");
module.exports = (agent) => {
    agent.customizeCollection('profilesUser', collection => {
        collection.addField('first_name', {
            columnType: 'String',
            dependencies: ['id'],
            getValues: async (records, context) => {
                return records.map(async record => {
                    return await fetchSubmissionItemValue(context, 9, record.id)
                });
            }
        });

        collection.addField('last_name', {
            columnType: 'String',
            dependencies: ['id'],
            getValues: async (records, context) => {
                return records.map(async record => {
                    return await fetchSubmissionItemValue(context, 10, record.id)
                });
            }
        });

        collection.addField('full_name', {
            columnType: 'String',
            dependencies: ['id', 'first_name', 'last_name'],
            getValues: async (records, context) => {
                return records.map(async record => {
                    return `${record.first_name} ${record.last_name}`
                });
            }
        });

        collection.replaceFieldOperator('first_name', 'IContains', (value, context) => {
            return {
                field: 'first_name',
                operator: 'IContains',
                value: value
            }
        });

    });
}

Context

  • Project name: Ourboro
  • Team name: Customer Experience
  • Environment name: Local
  • Agent technology: nodejs
  • Agent (forest package) name & version:
  • @forestadmin/agent: “^1.39.2”
  • @forestadmin/datasource-sql: “^1.14.0”
  • @forestadmin/datasource-toolkit: “^1.33.0”
  • Database type: Postgres
  • Recent changes made on your end if any: -
2

Hello @morekaccino,

Thanks for your very detailed message about the thing you’re trying to achieve.

Indeed, emulating search or filtering is very slow, and even slower in your case. The code does all the operations locally after retrieving all the values.

You correctly guessed that you needed to use replaceFieldOperator in order to let people search and filter by first_name and last_name.

For information, in order for the fields to be filterable, you’ll need to implement all these minimal operators:

  1. In, which takes an array of values
  2. NotIn
  3. Like
  4. NotContains

In order for the fields to be searchable, implementing IContains can make the search experience a little better.

To implement these operators, you need to retrieve a list of ids that correspond to the given criterion, and then return a filter on these ids in your operator.

For instance:

collection.replaceFieldOperator('first_name', 'IContains', async (value, context) => {
  const matchingIds = await callYourAPIHere(value, 'IContains');

  return {
    field: 'id',
    operator: 'In',
    value: ids
  }
});
3

It works! thank you!

Also just wanted to make a note of this error in the documentation here:

“aggregation” needs to be replaced with “aggregator” otherwise the agent returns an error

4

Thanks a lot for your feedback on the documentation. I fixed the example according to your suggestion.

Nice to see that everything is working as expected.