Running django-forestadmin application on google app engine fails when DEBUG=True

Help me!
1

Feature(s) impacted

django-forestadmin backends deployed on Google app engine

Observed behavior

We created a django app which works locally and works well with forestadmin in development environment. However, when we deployed this to our staging environment, we encountered a problem and when we attempted to get more information by enabling DEBUG=True to get more info from django, the app could no longer start up.

Digging into this we discovered that when debugging is enabled, it attempts to write the current working directory as seen here:

However, on Google App engine, the working directory is a read-only filesystem and an exception is thrown.

I forked this library on github to disable the writing and that solved the problem, however, I’d like to avoid using my hacked version long term. Seems like this could be avoided by allowing the location of the .forestadmin-schema.json be overriden somehow.

I’m also unclear what this file is used for and how we should treat it. It looks like it’s something that’s generated when we run in debug mode, so is the idea that we should run our app, generate the file, and switch the debug = False, and then deploy the file we got from running it locally?

If that’s the case I think I’d like to add the generation of this file into our deploy script to make sure it’s always up to date. However, I’m unclear if that’s how this file is intended to be used. It seems a little odd to have this dependency on a file which is generated from running the app in debug mode.

Expected behavior

The application to start.

Context

  • Project name: GumboUI
  • Team name: Operations
  • Environment name: staging
  • Agent type & version: django-forestadmin
2

Hello @pmo,

  1. The .forestadmin-schema.json is generated to describe your models in order to create the linked collections.
  2. It should be generated in your dev environment and versioned in your own repository.
  3. It is created when you start your application, thanks to the init_forest(), and when your settings.DEBUG == True. It should be always up to date thanks to the dev django server.
  4. It is mandatory in your production environment

I have not a clear fix for your specific issue. In my case, I’ll set the settings.DEBUG to False in the staging environment to be uniform with the prod environment but I have not all the cards.

Valentin,

3

Thanks that’s helpful.

I think the issue we encountered was that when we deployed to our staging environment (which was hosted on google app engine) we started getting an error that we didn’t have locally.

We then set DEBUG=True in our staging environment to get additional logging from django. (The problem turned out to be a django configuration issue not directly related to the forest admin agent). However, turning DEBUG=True on app engine resulted in a new error as forest admin attempted to regenerate .forestadmin-schema.json and store it on read-only filesystem.

Regardless, thanks for clarifying what the lifecycle of this file is supposed to be. That’s helpful.

I’ll see if we can make a minimally invasive pull-request in our fork that we can submit back to you for review to avoid attempting to the write-only filesystem. (The least invasive solution I can think of would be to add a config parameter which could be used to override the path used for .forestadmin-schema.json. If you think that’s a bad idea, please let me know.)

Thanks,
Phil

5

What is the new error ? If you push the generated file in your server it should be works fine.

6

The error was an exception was thrown on this line of code:

 with open(file_path, 'w') as f:

because on google app engine, the entire filesystem is read-only. ( see Python 3 Runtime Environment  |  Google App Engine standard environment docs  |  Google Cloud for documentation which states:

Filesystem
The runtime includes a full filesystem. The filesystem is read-only except for the location /tmp, which is a virtual disk storing data in your App Engine instance’s RAM.
)

However, my understanding is that this file is generated on the fly so it’s basically a temp file so when deploying to app engine, I’d like to write it to /tmp/.forestadmin-schema.json

That’s why I proposed a config setting which would allow me to override where this file is written/read from.

7

In case it’s helpful, I took a look at what forest-express does, and found there a setting like I’m proposing.

I see that library is using a setting called “schemaDir” to construct the path to the file:

And based on this thread, it sounds like it works the way I’m hoping (controls where its read/written):

So, I’m proposing we add an analogous setting in the django version. It’s a trivial change, so I’m happy to add it, but I wanted to check if you think that sounds reasonable before doing so.

8

Hi @pmo,

Sorry, I didn’t have a notification for your answers. Sounds good to me. Feel free to build a PR.