Custom Filters for Smart Charts on new Node.js agent

[David_Roegiers]

Feature(s) impacted

I am building a Smart Chart, where the user can select the time period. Inspired by this comment:

Filters on smart chart

Ok, sure it is possible. You just have to implement the filter you describe by yourself (this is not provided by Forest). For example: Template code: add some controls to select the timeperiod Component code: get the timeperiod from the controls and pass it the fetch request Route code: get the timeperiod parameters from the request and pass them to the transactions.find(...) call. Best regards

When the time period has been selected and the user presses a button, in the custom frontend. I call the Smart Chart forest/_charts/chart-name, as below. The plan is to inject the time period, which was selected in the UI, as a parameter to the lianaServerFetch call. No documentation exists online, which explains the API of lianaServerFetch and thus, how this can be achieved.

Observed behavior

I can’t find a way to pass a parameter to the call /forest/_charts/chart-name. I tried with test: true, but I can not retrieve this in the addChart callback on the server.

export default class extends Component {
  @service lianaServerFetch;

  constructor(...args) {
    super(...args);
  }

  async load() {

    // Load data from agent
    const response = await this.lianaServerFetch.fetch(
      '/forest/_charts/chart-name',
      {
           test: true;
      },
    );

Expected behavior

I would expect to be able to pass some parameters from the frontend, in order to make smart charts interactive, not sure if it has been built in yet. Any guidance on the topic would be greatly appreciated.

Failure Logs

No failure logs.

Context

• Project name: jurata-2
• Team name: jurata
• Environment name: Development

"@forestadmin/agent": "^1.27.1",
"@forestadmin/datasource-mongoose": "1.5.15",
"@forestadmin/datasource-replica": "^1.0.16",
"@forestadmin/plugin-flattener": "^1.0.53",

[anon34731316]

Hello @David_Roegiers,

Thanks for your message!

Can you try to use it that way (base on this documentation) and tell me if it works?

const response = await this.lianaServerFetch.fetch(
      '/forest/_charts/chart-name',
      {
        method: 'POST',
        body: JSON.stringify({
          test: true;
        }),
      },
    );

Cheers!

[David_Roegiers]

Thanks, @anon34731316. I will give this a try today.

[David_Roegiers]

Hi @anon34731316

This didn’t work, unfortunately. I got the following feedback:

chunk.644.c2a2670….js:2 Uncaught (in promise) TypeError: Failed to execute 'fetch' on 'Window': Request with GET/HEAD method cannot have body.
    at An.bn (chunk.644.c2a2670bf27efce0e548.js:2:37245)
    at i (chunk.644.c2a2670bf27efce0e548.js:2:33783)
    at o.fetch (client-06341e2325a46033ec32754e303e3298.js:1:4376989)

That makes sense, as it is a HTTP GET request. I could send it out using URL query parameters, but I’m not sure how to extract them in the addGraph function?

[David_Roegiers]

@anon34731316

So I can pass URL query parameters in the request, but I don’t find a way to access this data, within this function:

agent.addChart('chart-name', async (context, resultBuilder) => {
    ...
});

Is there a quick way to expose this data (as was done with the caller.requestId)? Just want to point out that this is - at least for us - a very valuable feature. To give some context: dashboards to analyze and interpret data from our product and business, are effective, only if we can manipulate filters and time periods. With ember, this has been made possible on the frontend, but not being able to read such parameters in the backend (for the new Node.js agent, not the older ones) is a big limitation and blocker for us. It would suck, if we need to find another tool for data dashboards.

Thanks for your consideration.

[anon34731316]

Hello @David_Roegiers,

Thanks for your answer.

I’m going to deep a bit more into this subject, and I’ll get back to you.

Thanks!

[anon34731316]

Hi @David_Roegiers,

I’m getting back to you.

So, I totally understand your request.
Now, could you tell me in what Forest context are you trying to create this chart? In Dashboards? Analytics (within a record in Data)? Or Workspaces?

If in Dashboards or Analytics, you’re totally right, there is something here on our side and you’re not able to get the query params We definitely need to work on something here.

In Workspaces context, if you’re ok working with existing charts (not smart or API ones), then the workspace context is given in the request - and a new component Date Picker is just being started to be developed and should be live in about a month. So you’ll have native date picker component that will enable you to filter native chart component. Would that suit your need?

Let me know.
Cheers!

[David_Roegiers]

Hi @anon34731316

Thanks for your proactive understanding for this feature request.

Unfortunately I need smart charts, e.g. mixed bar abd line plots, and custom data aggregation on the backend. So accordibg to your description, Workspaces does not work for us.

[anon34731316]

@David_Roegiers,

Right, I see.
So, I’m opening a feature request as of now so that we’ll tackle this need in the future.

Meanwhile, you could implement a totally custom route of your own to make what you wish, right?

I’m sorry for the inconvenience.
Cheers,

[David_Roegiers]

Hi @anon34731316

OK, thanks.

Yes, a custom route would work. But, how do I address the context in a custom route? I would like to have access to all the ForestAdmin collections, which do a lot of data enrichment and transformation, that I can build on. Or is that not possible?

[anon34731316]

Hey @David_Roegiers,

Sorry for the delay.

We’ve talked about your subject with the team: unfortunately, we don’t have a great solution to suggest here. You won’t be able to access easily to all the ForestAdmin collections
Of course, you could always do everything from scratch in a custom route, but I totally understand that’s not a very attractive solution.

Anyway, rest assured your point is taken.
We’ll let you know once we’ve build a better experience for this need.

Thanks again for bringing that up!
Cheers.

[David_Roegiers]

Hi @anon34731316

Thanks for your reply.

That’s really unfortunate. We’ve invested quite some time into getting all our backend data aggregated, enriched, and ready, using the ForestAdmin collections. This is a fundamental data layer to base our operations, analytics, and other data-related activities on.

The use case that we are discussing, i.e. manipulating graph filters, was possible with the former ForestAdmin agent. Moreover, the Metabase integration does not help here: as I understand, it is an iFrame into the ForestAdmin dashboard, but does not build on top of the data produced by the ForestAdmin collections.

Given the big red banner warning, which FA shared in our dashboard, I pushed forward with the migration, assuming, that it would not lose this capability.

That might have been a bit too quick. Now we are stuck.

Anyway, rest assured your point is taken.

I appreciate your acknowledgement of my point, but I wonder if this is going straight to the product management parking lot, and stay there for … well, a long time?

Or do you not have the possibility to sneak a simple URL Query Parameter exposure in the caller object, such as was done with requestId. It might be a small hack (at least from where I am sitting), offering a plethora of analytics functionality to your users.

I don’t want to overstep the mark here, but what do you say?

[arnaud]

Hi @David_Roegiers

The team understands your frustration.

I am not really sure about the “red banner warning” has something to do with the migration to the new generation of agent. An upgrade to the latest old generation was enough to use a version that is still supported and make the warning banner disappear.

That being said, I don’t really know if it is a new use case or if it was something you had with your previous agent implementation but I believe that it would have to be implemented the same kind of way in the old or new agent generation; fully manual implementation.

I am not sure to get what you mean by “manipulating graph filters, was possible with the former ForestAdmin agent”. With what type of charts do you think it was possible and what do you call “filters”?

How dynamic your charts have to be?
It is not really clear to me what exact context you need for these computations.

Being able to pass parameters to an API chart to change it’s computation and result, is a request that is really new to us as it implies implementing a Smart Charts using some API Charts implemented on the agent that are not static. And I am afraid that, so far, you’re the only company requesting for this.

Anyways, we need to mature this request as a team to understand how we could respond to it, but it will probably take time to open the right gate.

[David_Roegiers]

I am not really sure about the “red banner warning” has something to do with the migration to the new generation of agent. An upgrade to the latest old generation was enough to use a version that is still supported and make the warning banner disappear.

Upon seeing the need for migration work, I jumped to the latest version, in order to make bigger advancements. I was not aware the new agent, was as new as it turns out to be. Anyway, my approach thus leads to issues as the current one.

but I believe that it would have to be implemented the same kind of way in the old or new agent generation; fully manual implementation.

To my understanding with the old agent, I would have access to the smart fields in the endpoints, where I was preparing the chart data,

I am not sure to get what you mean by “manipulating graph filters, was possible with the former ForestAdmin agent”. With what type of charts do you think it was possible and what do you call “filters”?

From the issue description, I want to select the time period over which the data is displayed. Or select a segment of the data: for example, do a funnel analysis for all lawyers (or only those belonging to a firm, which has more than 10 people), which is a data point exposed by an enumerated smart field.

How dynamic your charts have to be?

Upon submitting a change to the filter, once, I press a button. See the attached image, for the frontend prototype I created for this.

Being able to pass parameters to an API chart to change it’s computation and result, is a request that is really new to us as it implies implementing a Smart Charts using some API Charts implemented on the agent that are not static. And I am afraid that, so far, you’re the only company requesting for this.

Sorry, the certainty of your colleague @Sliman_Medini’s comment made me assume otherwise.

Filters on smart chart

Ok, sure it is possible. You just have to implement the filter you describe by yourself (this is not provided by Forest). For example: Template code: add some controls to select the timeperiod Component code: get the timeperiod from the controls and pass it the fetch request Route code: get the timeperiod parameters from the request and pass them to the transactions.find(...) call. Best regards

If we are the only company, who needs this, which surprises me, I fully understand, that you will not dedicate a feature solely to such a small fish as us. I appreciate the transparency, which confirms to me, that I should search for another solution.

Thanks for the chat and have a good evening!