Map smart view with millions of points, dynamic display on zoom

[Elo]

Feature(s) impacted

BavAR[t] project includes millions of points representing arts to capture, distributed throughout France. Each point is an Art Instance (forest collection).

We would like to display these Art Instances on a map and not a table for easier management.

I applied this guide :

Create a Map view | v9 (v1 Django/Laravel)

Observed behavior

I succeeded displaying the Art Instances on map view but it has limitations that makes it unusable in our context :

• While creating or editing a smart view, records per page is limited to 500 maximum (see picture of triggered error below). But when applying map view, it is impossible to switch pages and therefore see all the data.

• Loading millions of points on the map would be too heavy anyway.

Expected behavior

We would like to load Art Instances depending on the level of zoom on the map. Above a certain threshold oh zoom, points will be clustered. If you zoom enough you will see the clickable Art Instances. This is how Art Instances are displayed on the app.

Clusters of points (can’t be clicked)

Art Instances appear if you zoom enough, and on click would redirect to single Art Instance forest page

We thought of using our API to fetch arts instances depending on the zoom level on the leaflet map. Is it possible to implement this in a Smart View ? We encountered already some CORS issues while trying to fetch API from Smart View (see failure log below). Is it possible to debug this only in forest smart view, without modifying the server ?

Thanks for any help !

Failure Logs

• Error message for changing Smart View records per page above 500 :

ArtInstancesMap update failed: Invalid patch value (path: "/collections/artInstance/layout/viewLists/e8b40289-15d0-4c47-9d98-3526dff9f222/recordsPerPage", op: "replace", value: 501): ValidationError: "value" Invalid number of records per page.

• Error message when trying to fetch from our API :

Blocking a multi-origin request (Cross-Origin Request): the “Same Origin” policy does not allow consulting the remote resource located at https://backend-prod.bavart.io/api/v1/arts/searchLight?latitude =47.87502&longitude=-3.92245&radius_meters=1000. Reason: The CORS header "Access-Control-Allow-Origin" is missing. Status code: 405.

Context

• Project name: BavAR[t]
• Team name: AR[t] Studio
• Environment name: Development and Production
• Agent technology: nodejs
• Agent (forest package) name & version: forest-express-sequelize v9
• Database type: postgreSQL
• Recent changes made on your end if any: none related to this issue

[Nicolas.M]

Hello @Elo,

It should indeed be possible to use your own API rather than the forest agent to fetch your map data. However, without specific configuration on your own server, you will indeed run into CORS errors, since the headers returned prevent data from getting fetch from within another domain (app.forestadmin.com in this case).
You may try to dev/debug the case without changing anything on your server by disabling cors checks in your browser (here for chrome for instance), but note that this will expose your development browser to security holes.

I hope this helps

[ChloeG]

Hi @Nicolas.M !
Thanks a lot for your reply! I’m working with @Elo on this, so I’m answering here.
The second link “Setup CORS” is the same as the first one, can you please share with us the correct one ?

Other question : is there a way to change pages when we are using map view ? One of our issues is also is that we can change the number of record per page, but once in map view we are not able to change the page, as usual when you have a record list.

Thanks a lot!
Have a nice day,
Chloé

[Elo]

Hello @Nicolas.M,

Thank you for your quick answer. To be clear about CORS : if i understand correctly, we need to do the configuration on our server where the API runs ? And not on forest side (front or back) ?

Thanks!

[Nicolas.M]

Hello,

Yes, indeed, the CORS setup is to be done on your own API, to let it know it can accept requests from domain app.forestadmin.com

Sorry but actually the CORS documentation that I was thinking about only concerns Ruby on Rails.
If you are using express, the config in your API would look something like this:

const express = require('express')
const cors = require('cors')
const app = express()

app.use(cors({
  origin: [
    'https://your.app.domain.com/*',
    'https://app.forestadmin.com/*'
  ],
}));

Guide
You may need to adapt the configuration to your own use case and libraries

If you still encounter CORS errors in your browser, check in your dev tools which URL triggered them and consider adding them to your CORS exceptions.

Note: ensure that your own frontend will still be able to access your app !!!

[Elo]

We finally succeeded to implement this map view ! We used in our case the Fast API documentation for CORS middleware.

Good to know for anybody trying to do a similar thing : if you restrict methods, don’t forget to add aswell OPTIONS method, which is sent in https.

app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["GET", "OPTIONS"],
    allow_headers=["*"],
)

Thank you for your help !