it seems like the culprit has been found and the fix is done, and is currently about to be released. We need to run a migration internally though, so @anon37102731 has to first release it on staging, ensure there is a green light and then deploy it to production (this is the reason why it is requiring a little more time than usual). However at this point the fix should land very soon.
It was a very nasty bug, @anon37102731 will be posting more details about what happened as soon as the fix has landed.
We just ran the migration in production and everything should be back to normal for any of your admin panels! Can you confirm this solves the problem for you?
For those of you who are wondering what happened exactly, here is a small overview:
Our environment removal service had a small flow that could lead to some models being related to dropped environment. This flow seems to have existed for quite a while but we recently released a small update on one of our front-end component that blocked the access on projects related to those deleted environments.
As for why we didn’t rollback, it’s because we were almost ready to release a fix that would patch the “deep” issue instead of reverting changes that were not directly related.
Nevertheless, thanks for your feedbacks and patience!