🚨 Forest down (SSL issue) - Trouble logging in - error 400: Invalid client id 🚨

I have a value set for ForestLiana.application_url in my initializer, which is my server url (https://api.okarito-internal.com/). :slight_smile:

Then, assuming you did not override the FOREST_URL env variable - you are calling a server with a valid SSL cert. (I see you are in a shell - I assume that’s the forest admin server shell).

In that case - I almost used up all my knowledge - the call to validate the code should work without failing on the expired cert. I will wait for the Forest experts to help here.

Sorry buddy!

1 Like

I suspect your Ruby runtime somehow has a cached SSL cert of the expired one - and the restart of the server did not clear it up. I am not a. Ruby expert by no means - but this is one direction I would check.

I’m digging in that direction indeed. Restarting my ECS docker instances didn’t clear the cache so I don’t believe it is cached in Ruby. Might be cached somewhere else. I’ll let you know.

Same problem here :confused:
I tried update backend URL but not have success

1 Like

We’re still experiencing the same issue after having re-generated the client IDs and restarting the servers.

Is there any information we can provide to help debug the issue further? We currently have all of our admin team locked out of ForestAdmin and would like to address this ASAP. Thanks

1 Like

@sergior @Douglas_Lara I would like to know:

  • Are you both running Forest admin on a Ruby server? (Which OS?)
  • Are you getting the same error as @remi_okarito? If not - what error do you get on the network tab calling *your own Forest admin server under /authentication/callback?

Hello everyone,

From what we understand the issue you are encountering is related to a rails dependency called HTTP client (issue can be found here).

Following the suggestion in the link above should resolve the issue

N.B: Some of our client followed the fix and are now able to access their production environment.

N.B.2: We are right now working on a fix for forest-rails and we will tell you as soon as it is available

3 Likes

Thanks for help @Yoad_Snapir @sergior @lclisson

This adjustment resolved :grinning:

2 Likes

My Rails app is deployed on Heroku using forest_env_secret and forest_auth_secret.
I restarted my dynos.
The problem still occurs, with an error 500:

2021-09-30T17:30:55.110113+00:00 heroku[router]: at=info method=OPTIONS path="/forest/authentication" host=my-splendid-app.herokuapp.com request_id=90647258-e981-47e7-a9e8-117246489536 fwd="REDACTED_IP_ADDRESS" dyno=web.1 connect=0ms service=14ms status=200 bytes=515 protocol=https
2021-09-30T17:30:55.099436+00:00 app[web.1]: I, [2021-09-30T17:30:55.099368 #4]  INFO -- : [90647258-e981-47e7-a9e8-117246489536] Started OPTIONS "/forest/authentication" for REDACTED_IP_ADDRESS at 2021-09-30 17:30:55 +0000
2021-09-30T17:30:55.226344+00:00 app[web.1]: I, [2021-09-30T17:30:55.226262 #4]  INFO -- : [04783205-736a-41f1-822c-a7a038e0e36a] Started POST "/forest/authentication" for REDACTED_IP_ADDRESS at 2021-09-30 17:30:55 +0000
2021-09-30T17:30:55.227811+00:00 app[web.1]: I, [2021-09-30T17:30:55.227738 #4]  INFO -- : [04783205-736a-41f1-822c-a7a038e0e36a] Processing by ForestLiana::AuthenticationController#start_authentication as */*
2021-09-30T17:30:55.227872+00:00 app[web.1]: I, [2021-09-30T17:30:55.227839 #4]  INFO -- : [04783205-736a-41f1-822c-a7a038e0e36a]   Parameters: {"renderingId"=>"87409"}
2021-09-30T17:30:55.229791+00:00 app[web.1]: I, [2021-09-30T17:30:55.229736 #4]  INFO -- : [04783205-736a-41f1-822c-a7a038e0e36a] Completed 200 OK in 2ms (Views: 0.3ms | Allocations: 399)
2021-09-30T17:30:55.234835+00:00 heroku[router]: at=info method=POST path="/forest/authentication" host=my-splendid-app.herokuapp.com request_id=04783205-736a-41f1-822c-a7a038e0e36a fwd="REDACTED_IP_ADDRESS" dyno=web.1 connect=0ms service=9ms status=200 bytes=2021 protocol=https
2021-09-30T17:30:55.506749+00:00 heroku[router]: at=info method=OPTIONS path="/forest/authentication/callback?code=REDACTED_TOKEN&state=%7B%22renderingId%22%3D%3E87409%7D" host=my-splendid-app.herokuapp.com request_id=acc6cfe2-15ea-4bb0-ae26-a069a1d54ea3 fwd="REDACTED_IP_ADDRESS" dyno=web.1 connect=0ms service=5ms status=200 bytes=515 protocol=https
2021-09-30T17:30:55.502663+00:00 app[web.1]: I, [2021-09-30T17:30:55.502590 #4]  INFO -- : [acc6cfe2-15ea-4bb0-ae26-a069a1d54ea3] Started OPTIONS "/forest/authentication/callback?code=REDACTED_TOKEN&state=%7B%22renderingId%22%3D%3E87409%7D" for REDACTED_IP_ADDRESS at 2021-09-30 17:30:55 +0000
2021-09-30T17:30:55.548686+00:00 app[web.1]: I, [2021-09-30T17:30:55.548623 #4]  INFO -- : [5daa4551-22a2-4453-91a3-0422eac6e35e] Started GET "/forest/authentication/callback?code=REDACTED_TOKEN&state=%7B%22renderingId%22%3D%3E87409%7D" for REDACTED_IP_ADDRESS at 2021-09-30 17:30:55 +0000
2021-09-30T17:30:55.552107+00:00 app[web.1]: I, [2021-09-30T17:30:55.551979 #4]  INFO -- : [5daa4551-22a2-4453-91a3-0422eac6e35e] Processing by ForestLiana::AuthenticationController#authentication_callback as */*
2021-09-30T17:30:55.552220+00:00 app[web.1]: I, [2021-09-30T17:30:55.552165 #4]  INFO -- : [5daa4551-22a2-4453-91a3-0422eac6e35e]   Parameters: {"code"=>"REDACTED_TOKEN", "state"=>"{\"renderingId\"=>87409}"}
2021-09-30T17:30:55.621418+00:00 app[web.1]: I, [2021-09-30T17:30:55.621327 #4]  INFO -- : [5daa4551-22a2-4453-91a3-0422eac6e35e] Completed 500 Internal Server Error in 69ms (Views: 0.3ms | Allocations: 836)
2021-09-30T17:30:55.627134+00:00 heroku[router]: at=info method=GET path="/forest/authentication/callback?code=REDACTED_TOKEN&state=%7B%22renderingId%22%3D%3E87409%7D" host=my-splendid-app.herokuapp.com request_id=5daa4551-22a2-4453-91a3-0422eac6e35e fwd="REDACTED_IP_ADDRESS" dyno=web.1 connect=0ms service=80ms status=500 bytes=913 protocol=https

Hello guys, it worked on our side.

I created a file inside config/initializers

# /config/initializers/http_client.rb

require 'httpclient'

class HTTPClient
  alias original_initialize initialize

  def initialize(*args, &block)
    original_initialize(*args, &block)
    # Force use of the default system CA certs (instead of the 6 year old bundled ones)
    @session_manager&.ssl_config&.set_default_paths
  end
end

and released the app again, I could login to my production env again.

5 Likes

Hi, I can confirm the fix used by @remi_okarito has worked for us as well and the users are able to login again.

3 Likes

@remi_okarito thank you for sharing the fix and @sergior glad to hear it worked also on your end.

@pil0u have you tried to monkey patch the httpClient library? See Connection to Lets Encrypt secured server fails Β· Issue #445 Β· nahi/httpclient Β· GitHub

I think I understand the underlying issue but I don’t really understand why it broke ForestAdmin at that particular moment, and why I should add code to my apps to fix it.
Is there any future changes on your side that would fix the problem avoiding me to add code for that, even if it’s not tonight?

thanks a lot @remi_okarito ! It worked for us too :slight_smile:

Hi everyone,

The origin of the issue you guys experimented with the Forest Admin gem is due to the failure of a pretty commun Ruby gem named httpclient the forest_liana uses.

This dependency contains an hardcoded list of CA certificates (including Let’s Encrypt ones), and unfortunately, today was the expiration date of the Let’s Encrypt DST Root CA X3 certificate. This lead to connection failures on the Forest Admin platform (as many other Ruby on Rails projects might have experienced around the world).

More info about that in the dedicated GitHub issue:

The first quick and dirty solution is to apply a Monkey Patch to your Rails app as mentioned above:

# /config/initializers/http_client.rb

require 'httpclient'

class HTTPClient
  alias original_initialize initialize

  def initialize(*args, &block)
    original_initialize(*args, &block)
    # Force use of the default system CA certs (instead of the 6 year old bundled ones)
    @session_manager&.ssl_config&.set_default_paths
  end
end

We immediately applied this monkey patch to our latest major versions to let you upgrade you Forest Admin gem and avoid the monkey patch addition on your own project:

The team is really sorry for the inconvenience and will take the time to work on ways to prevent similar issues in the future.

3 Likes

Wow, blazing fast! Thanks for the fix, I bumped to 7.2.2 and as expected, access is granted again.
Thanks team :+1:

Still not working to us with 7.2.2, in the logs I see:

{"errors":[{"status":400,"detail":"Invalid client id","meta":{},"name":"InvalidClientIdError"}]}

But we don’t have a static FOREST_CLIENT_ID

Hi @Maicol_Bentancor,
Did you restart your server as suggested by @louis in this message?

Yes, and even set the FOREST_CLIENT_ID just in case, but no luck