Cannot display my data (using JSONB columns)

Help me!
1

(EDIT: See my answer below - The actual blocking issue doesn’t seems to be related to pgvector, but to JSONB support)

Feature(s) impacted

Database introspection is not working on `vector(n)` columns - Columns are generated as string in `.forestadmin-schema.json`

(To be honest, this is what I suppose as failing - the rest of my table is pretty straightforward)

Observed behavior

image
image3024×1468 303 KB

Only one record is displayed - And I cannot click on it.
Column is generated as string in `.forestadmin-schema.json`

Expected behavior

pgvector columns are displayed, and the app is able to display my records

Failure Logs

image
image1250×350 57.5 KB

Context

Here is my DDL for this table (Using pgvector)

CREATE TABLE documents (
    id text PRIMARY KEY,
    source text NOT NULL,
    content text NOT NULL,
    content_hash text NOT NULL,
    embedding vector(1536) NOT NULL,
    metadata jsonb DEFAULT '{}'::jsonb,
    created_at timestamp without time zone DEFAULT now(),
    updated_at timestamp without time zone DEFAULT now()
);
  • Project name: demo-rag-generator
  • Team name: Operations
  • Environment name: Development
  • Database type: Postgres
  • Recent changes made on your end if any: …

And, if you are self-hosting your agent:

  • Agent technology: NodeJS (agent-nodejs)
  • Agent (forest package) name & version (from your .lock file): latest - I’ve just created the project.
1 Like
2

Funny enough, doesn’t seems related to pgvector.

I tried:

agent.addDataSource(

  createSqlDataSource({

uri: process.env.DATABASE_URL,

schema: process.env.DATABASE_SCHEMA,

sslMode: process.env.DATABASE_SSL_MODE,

  }),

).customizeCollection('documents', d => d.removeField('embedding'));

and the page is still failing to load all my records.

Removing metadata seems to fix it though, so my best guess would be that it’s metadata jsonb DEFAULT '{}'::jsonb, that is causing the issue here - Even though the column seems to be correctly generated in .forestadmin-schema.json :thinking:

I also tried removing my metadata field & added a new metadata_json column as follow

CREATE TABLE documents (
    id text PRIMARY KEY,
    source text NOT NULL,
    content text NOT NULL,
    content_hash text NOT NULL,
    embedding vector(1536) NOT NULL,
    metadata_json json,
    created_at timestamp without time zone DEFAULT now(),
    updated_at timestamp without time zone DEFAULT now()
);

and this seems to “fix” the issue - at least it displays all my records.

So, just to conclude:

  • embedding as vector(1536) is detected as String - which is not correct but not preventing the app to work as expected.
  • metadata as jsonb default to {} is detected as Json but prevents the frontend from displaying all my records.

Let me know if that helps.

3

Hi @Jeff_Ladiray :waving_hand: thank you so much for your feedback.
Thank you very much for the details you have provided.
I will personally take care of your issue. I will keep you informed as soon as I have reproduced and resolved your issue.

Kind regards.

1 Like
4

Hi @Jeff_Ladiray, we have released a fix on the app to display correctly your jsonb column.
Please let me know if that fix your issue. :folded_hands:

2 Likes
5

Works as expected, thanks :folded_hands:

1 Like