Upgrading to v7, MongoDB/Typescript documentation might be a bit obsolete

Hello,

I freshly updated my project from v4 to v7 (recreated a fresh project and reimported my code as it got way to hard xD)

However, as I prefer typescript, my models/routes are in typescript and… I have a bit of a problem…

Based on this documentation, regarding the models part: Translate your project into TypeScript - Woodshop

As we can see, you give an example that seems to be v5/v6:

const mongoose = require('mongoose');

const schema = mongoose.Schema({
  'firstName': String,
  'lastName': Date,
}, {
  timestamps: false,
});

module.exports = mongoose.model('users', schema, 'users');

This is easely translatable into typescript with the following:

import { Schema, Document, model} from 'mongoose';

interface IUser extends Document {
    firstName: string;
    lastName: string;
}

const schema = new Schema({
    'firstName': String,
    'lastName': Boolean,
}, {
    timestamps: false,
});

export default model<IUser>('users', schema, 'users');

However, in v7 (example reproducible by creating a fresh new project), the base model in js will look like so:

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models

module.exports = (mongoose, Mongoose) => {
  // This section contains the properties of your model, mapped to your collection's properties.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const schema = Mongoose.Schema({
      'firstName': String,
      'lastName': Boolean,
  }, {
      timestamps: false,
  });

  return mongoose.model('users', schema, 'users');
};

And… if you translate it into typescript, from my point of view, it should give something like this:

// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models
import {Document, Schema} from "mongoose";

export interface IUser extends Document {
    firstName: string,
    lastName: boolean,
}

// This section contains the properties of your model, mapped to your collection's properties.
// Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
const schema = new Schema({
    'firstName': String,
    'lastName': Boolean,
}, {
    timestamps: false,
});

//// Not sure about this
// export const collection = mongoose.model<IUser>('users', schema, 'Users');

module.exports = (mongoose, Mongoose) => {
    return mongoose.model('users', schema, 'users');
};

And here the problème starts… if I want to get a record by ID, then new RecordGetter(...) will do the job. However, when I want to import/findOne it doesn’t work… Either the default exports doesn’t have .findOne(...) method, either I try to export my collection/model manually:

export const collection = mongoose.model<IUser>('users', schema, 'Users');

and then import it like import {collection as usersCollection} from '../models/users.ts' but then, I get:

(node:252) UnhandledPromiseRejectionWarning: TypeError: Cannot read property ‘findOne’ of undefined

I really have no idea how to deal with that situation… I think first that I am missing something, of course, but I do feel as a second point that the documentation is obosolete and doesn’t show a croncret working example for the v7 when trying to make advanced queries from routes like shown at this link: Override a route - Documentation

Of course, assessors as RecordGetter, RecordUpdater, etc are good, but sometimes, updateMany, findOne, etc can do advanced stuff you know, so it’s good to keep their way of uses explained through your documentation :slight_smile:

Expected behavior

From my routes/users.ts, I want to (for some reason) query my collection (... from '../models/users.ts') using operation such as:

  • findOne(…)
  • updateMany(…)
  • etc
    instead of using RecordGetter, RecordRemover, etc

Actual behavior

I can’t find the way to access such operation from the default function exported or the collection itself using a Typescript approach

Failure Logs

  • When exporting the collection as above:
import {collection as usersCollection} from "../models/users";
// ...
function foo() : IUser {
    // ...
    usersCollection.findOne({...})...
}

(node:252) UnhandledPromiseRejectionWarning: TypeError: Cannot read property ‘findOne’ of undefined


  • When trying to access the default export’s findOne() method:
import * as users from "../models/users";
// ...
users.findOne(...)

TS2339: Property ‘findOne’ does not exist on type ‘typeof import(“C:/Workspace/…/models/users”)’.


const { users} = require('../models');
// ...
const recordGetter = new RecordGetter(users);

TS2305: Module ‘"…/models"’ has no exported member ‘users’.

Context

Please provide any relevant information about your setup.

package.json

{
  "name": "project-backoffice",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node ./dist/server.js",
    "legacy-start": "node server.js",
    "build": "tsc",
    "start-dev": "tsc-watch --project ./tsconfig.json --noClear --onSuccess \"nodemon --watch ./dist ./dist/server.js\""
  },
  "dependencies": {
    "@types/express": "^4.17.11",
    "@types/forest-express-mongoose": "^7.5.0",
    "@types/mongoose": "^5.10.5",
    "@types/node": "^15.3.0",
    "axios": "^0.21.1",
    "body-parser": "1.19.0",
    "chalk": "~4.1.1",
    "cookie-parser": "1.4.5",
    "cors": "2.8.5",
    "debug": "~4.3.1",
    "dotenv": "~9.0.2",
    "express": "~4.17.1",
    "express-jwt": "6.0.0",
    "forest-express-mongoose": "^7.6.0",
    "mongoose": "~5.12.9",
    "morgan": "1.10.0",
    "nodemon": "^2.0.7",
    "require-all": "^3.0.0",
    "tsc-watch": "^4.2.9",
    "typescript": "^4.2.4"
  }
}

PS: IMPORTANT NOTES

/models/index.js conversion into typescript has issues

File example: Upgrade to v7 - Documentation

  • First, it can’t work as const db = {} doesn’t have any of objectMapping or connections defined, so the two of them must be added manually.
  • Second, theline .filter((file) => file.indexOf('.') !== 0 && file !== 'index.js') doesn’t work properly and loads .js.map files which creates some errors… The way to fix it is simply to replace the line by .filter((file) => !file.includes('.map') && file.indexOf('.') !== 0 && file !== 'index.js'

/models/*.js generation has empty schema

  • I don’t know if it’s the expected behavior, but, using latest NPM lumber version (as of today), when the models files gets created, they all have empty schema, is it normal? Only the timestamp part is present:
// This model was generated by Lumber. However, you remain in control of your models.
// Learn how here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models

module.exports = (mongoose, Mongoose) => {
  // This section contains the properties of your model, mapped to your collection's properties.
  // Learn more here: https://docs.forestadmin.com/documentation/v/v6/reference-guide/models/enrich-your-models#declaring-a-new-field-in-a-model
  const schema = Mongoose.Schema({
  }, {
    timestamps: false,
  });

  return mongoose.model('users', schema, 'users');
};

FOREST_AUTH_SECRET if I loose this variable, do I have a way to retrieve it?

  • I had an issue (fully my mistake) where I removed the project, but not the right one… As the .env file is not versionned… I had a little bit of a hard time retrieving it from another computer. When I recreated it manually, I was missing this auth secret and couldn’t find the way to get it from ForestAdmin dashboard

express-jwt version 6 support instead of v5.3.1 (at project generation)

PLEASE ADD FULL TYPESCRIPT GENERATION

I know, it’s a lot of work and you guys are doing really great, to be honest, I have a lot of respect for your work, your product, your reactivity etc. However, I big point which is missing is that you don’t have a full typescript version of your product and, even when you try to convert everything into typescript, you run into errors with stuff such as Liana, so please, put it on your roadmap, I am sure it would give you a lot of precious points!! :slight_smile:


Thank you so much and sorry for that big post!

Hope to hear from you soon! :slight_smile:

Best,

Max

Hi @Emixam23,

Thank you for reaching out and sharing your issue with us :slight_smile:

Regarding the usage of the RecordsGetter, it is used by us to add logic in queries such as handling properly the parameters, ensuring user rights to access data, etc. Bypassing it and implementing your own get would work but it’s not the preferred way. And unfortunately yes, we do not expose a getOne but it is still possible to do it through the getAll and returning the first element.

Also if you only want to make a get by id, forest-express-mongoose exposes a RecordGetter that has a get(recordId) method.

But in your case, when using the RecordGetter it seems that the models cannot be imported properly from the models folder. Could should share with me the content of the models/index.js please? :pray:

Thank you also for the feedback you provided on the fact that our generated code is not in Typescript. I do understand that some users, as yourself, consider TypeScript as the preferred way of developing in Javascript. But not all users are used to TypeScript and plain JavaScript still seems to us the best way to reach a wider range of developers (and not to mention that it would indeed require a lot of dev time to provide a fully supported TypeScript project generation).

However it’s important for us to keep track of those feedbacks to prioritize our future roadmap. I’ll definitely log it into our ProductBoard :+1:

Hello :slight_smile:

Alright then I am ok with what you said, but for some reason, I sometimes need to “getOne” but not based on the document ID. I actually query another parameter…

Also, for the update, I can also update using record updater maybe… I don’t know I need to check tbh… I just felt like it was easier to use directly mongoose but I can definetly try RecordUpdater! :slight_smile:

/models/index.ts

import * as fs from 'fs';
import * as path from 'path';
import * as Mongoose from 'mongoose';

const databasesConfiguration = require('../config/databases');

const connections = {};
const db = {
    objectMapping: undefined,
    connections: undefined
};

databasesConfiguration.forEach((databaseInfo) => {
    const connection = Mongoose.createConnection(databaseInfo.connection.url, databaseInfo.connection.options);
    connections[databaseInfo.name] = connection;

    const modelsDir = databaseInfo.modelsDir || path.join(__dirname, databaseInfo.name);
    fs
        .readdirSync(modelsDir)
        .filter((file) => !file.includes('.map') && file.indexOf('.') !== 0 && file !== 'index.js')
        .forEach((file) => {
            try {
                const model = require(path.join(modelsDir, file))(connection, Mongoose);
                db[model.modelName] = model;
            } catch (error) {
                console.error(`Model "${file}" creation error: ${error}`);
            }
        });
});

db.objectMapping = Mongoose;
db.connections = connections;

module.exports = db;

There is my index as you asked :slight_smile:

Hope it helps, thanks for your feedback :slight_smile:

Max

Hello @Emixam23 :wave:

First of all, thanks for the very complete and useful feedback :raised_hands: This will really help to improve the Typescript ecosystem at Forest.

I have investigated on this topic, and indeed, the document to translate projects from js into ts is not compatible for v7 projects (especially for mongoose projects).

I’m writing this message just to keep you posted, I will write a brand new documentation for v7 projects to workaround every pains / blocking points you pointed out above. Sadly I can’t provide any information for now, I have just started the investigation.

I will answer on this thread as soon as I have more information on this topic to unlock your situation.

Sorry for the inconvenience,

Looking forward to get back to you with good news.

Steve.

2 Likes

Hey @Steve_Bunlon! Hope you’re doing great :slight_smile:

Thanks for your message. At first, I’d like to apologize, I thought I responded to you, sorry about it.

Second, I’d like to thank you for your answer and I am glad my feedback could lead you to some future updates :slight_smile:

Third, regarding the update of your documentation/product, when do you think you could make it up? (no pressure, but… We are kinda stuck and we don’t want our project to stop to work on June 30th…)

Feel free to keep me updated because if the project will for sure stop working on the 30th of June, I need to prepare myself :slight_smile:

Best!

Max

Hello @Emixam23,

We are currently actively working on fixing all the Typescript related issues on our project. I cannot give you an estimation for now, but I am pretty confident it will be before the 30th of June. :crossed_fingers:

Please stay assured that your project won’t stop working after the 30th of June.

Let me explain to you what could happen after June 30th.

Only the development environment could be impacted and only if you will upgrade your Chrome browser to version 92 (you can find some details here about the changes from Chome side). As those changes will only affect the access to localhost, the production environment won’t be impacted in any way. :tada:

To sum up, even if you don’t upgrade the liana by the 30th of June, your main operations should not suffer from the upgrade of the Chrome version.

In case we are not able to provide you with clear documentation about Typescript and forest-express-mongoose v7, we have an intermediary solution for you. As it’s not the recommended approach I will not publish it right now, let’s keep it for a worst-case scenario.

We’ll keep you posted about our progress in translating a v7 project into typescript. I will let this thread open so we can update you here.

Please let me know if you have any other questions or concerns.

Regards,
Olesya

1 Like