WebMO - Computational chemistry on the WWW
Recent news

WebMO 19.0 is now available for free download!

WebMO 19.0 Pro and Enterprise have a variety of additional features and is available for purchase.

The WebMO app is now available for iOS and Android.

August 24, 2019

Rapid ssh connections problem Log Out | Topics | Search
Moderators | Edit Profile

WebMO Support Forum » Administering WebMO » Remote Server Manager (Pro) » Rapid ssh connections problem « Previous Next »

Author Message
Chris Willing
Unregistered guest
Posted on Thursday, June 13, 2013 - 2:18 am:   

Working through installation of 12.1.004e - server registration works using a test application server. However when attempting to register the real server (where passwordless ssh connection already works fine), registration fails. Subsequent manual ssh to the server now fails, where it had previously worked. I believe this is a firewall issue; the registration seems to involve many rapid ssh connections (I could see 27 in about 1 second) which I guess the firewall regards as a DOS attack.

Firstly, is this behaviour (many rapid ssh connections during server registration) as expected?

Secondly, if it is as expected then can it be changed? I'd rather have a longer registration process that is more likely to succeed (say 1 connection per second) rather than 27 connections per second which is bound to fail.

Thanks for any insight ..
chris
JR Schmidt
Moderator
Username: Schmidt

Post Number: 355
Registered: 11-2006
Posted on Thursday, June 13, 2013 - 10:07 am:   

The registration process DOES involve many SSH connections, as there are numerous file copies / command that need to be run (via SSH) to configure the remote server. However I count only ~15 not 27.

This should be less of an issue once the server is added. In the mean time, you might add a "sleep(1);" statement to the "remote_run_command" and "remote_copy_to" subroutines in "servercontrol.cgi" to effectively throttle the rate of SSH commands being issued.

REMOVE THESE STATEMENTS after registering the server(s).

Note that you are likely to have a MUCH more stable connection if you are not running a connection through a hardware firewall (e.g. to a server in the same building, etc.) because WebMO does require a stable connection. In my experience, using truly remote servers yields intermittent connection problems which can cause jobs to fail. (This is not a frequent issue, but does happen occasionally).
Chris Willing
Unregistered guest
Posted on Thursday, June 13, 2013 - 9:53 pm:   

It turns out (from other testing) that I needed a timeout of 20 seconds per ssh connection. I added sleep(20) in those subroutines you mentioned, however I'm not positively sure it has worked. On the one hand, the browser (after clicking the Add button) eventually came up with a "Gateway Timeout" error message i.e. looked like failure. However the server now appears in the "Edit servers" tab of the remote server manager i.e. looks like success.

I also see that a webmo directory has appeared on the server (with quite a bit in there); also 2 perl scripts (findfiles.pl and pscheck.pl), which had previously been there, presumably after earlier registration attempts failed to complete, are now gone - suggesting some cleanup has happened.

Is there any way to confirm that server registration has really completed successfully?

chris
JR Schmidt
Moderator
Username: Schmidt

Post Number: 357
Registered: 11-2006
Posted on Thursday, June 13, 2013 - 10:09 pm:   

The fact that the findfiles.pl and pscheck.pl are still there suggest that the addition of the server did NOT complete. The "Gateway Timeout" means that the web server terminated the script prior to completion since it took too long to complete (due to sleep 20!)

I don't have a good workaround for this. In principle this could be re-written to use fewer SSH calls, but I have never encountered this situation before!

If you add the server from a local network connection, it will likely work fine once it is added
Chris Willing
Unregistered guest
Posted on Friday, June 14, 2013 - 12:31 am:   

I meant that since adding sleep(20), despite the Gateway Timeout, those perl scripts have now been cleaned up (they were not cleaned up previously), so I thought things were looking good - I was just looking for some other confirmation.

Since then, looking through servercontrol.cgi I see that removing pscheck.pl is the last thing done in the add_server function so I'm regarding the absence of pscheck.pl as even more of a positive sign.

Better still, I've now changed the webserver configuration, adding "Timeout 600" to http.conf (I believe the built in default was 60), so now there is no Gateway Timeout error any more. I can monitor the application server and see the various files & directories as they're created (and eventually cleaned up).

Its not ideal but seems to work so far ..

Thanks for your help,
chris
JR Schmidt
Moderator
Username: Schmidt

Post Number: 358
Registered: 11-2006
Posted on Friday, June 14, 2013 - 10:06 am:   

This sounds good. Depending on the threshold for number of connections before the firewall freaks out, this may (or may not) cause problems when submitting jobs to remote servers. That ALSO requires multiple SSH connections, although far fewer then adding the server.

Add Your Message Here
Post:
Username: Posting Information:
This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Password:
E-mail:
Options: Post as "Anonymous"
Enable HTML code in message
Automatically activate URLs in message
Action:

Topics | Last Day | Last Week | Tree View | Search | Help/Instructions | Program Credits Administration