Scheduler request failed: HTTP internal server error

Message boards : Number crunching : Scheduler request failed: HTTP internal server error

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Profile Tim Turner
Avatar

Send message
Joined: 1 May 09
Posts: 570
Credit: 184,322
RAC: 0
Message 348 - Posted: 14 May 2009, 15:15:04 UTC - in response to Message 345.  


Expect: 100-continue



maybe this is the problem.

ID: 348 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Bok

Send message
Joined: 21 Apr 09
Posts: 5
Credit: 261,336
RAC: 1
Message 349 - Posted: 14 May 2009, 15:22:42 UTC

Do the apache logs not contain any hints ?

what is LimitRequestBody set to in the httpd.conf ?

Also, though I don't think it's anything to do with php, can't hurt to check the php.ini for upload_max_filesize perhaps..

Throwing some ideas out there :)
ID: 349 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Heavy Metal Dungeon Keeper

Send message
Joined: 2 May 09
Posts: 22
Credit: 81,216
RAC: 0
Message 350 - Posted: 14 May 2009, 15:22:57 UTC
Last modified: 14 May 2009, 15:25:10 UTC

I've checked the output for 8 other project updates and have found 1 difference:

On DD@H I get:

14/05/2009 16:07:40 [http_debug] [ID#0] Received header from server: HTTP/1.1 100 Continue

14/05/2009 16:07:56 [http_debug] [ID#0] Received header from server: HTTP/1.1 500 Internal Server Error

On the others I get a difference in the second line above:

14/05/2009 16:15:34 [http_debug] [ID#9] Received header from server: HTTP/1.1 100 Continue

14/05/2009 16:15:35 [http_debug] [ID#9] Received header from server: HTTP/1.1 200 OK

Where DD@H has a value of 500 and the others 200

Or it may just be where the internal server error message comes from
ID: 350 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 351 - Posted: 14 May 2009, 15:34:31 UTC - in response to Message 350.  

The number 500 is the error number for the HTTP Internal Server Error.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 351 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 352 - Posted: 14 May 2009, 15:36:28 UTC - in response to Message 346.  
Last modified: 14 May 2009, 15:38:35 UTC

him, if it doesn't get solved in the next day or so go to plan B:

have those who can't upload them, dl 7zip and compress their boinc drug data, and send it to jack.

It's not uploading that is the problem. All these tasks are already uploaded to disk space on the server. It is reporting them to the database that is the problem. So even sending complete client_state.xml files isn't going to help as that data can't be easily inserted in the database.

We could easily tell people to reset their project, and "give" credit for the missed tasks, but that isn't going to help us with that work. The reports contain data about the tasks, if they completed correctly or not. We want that data.

So do NOT reset your project. We're still working on a fix.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 352 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 354 - Posted: 14 May 2009, 16:04:37 UTC - in response to Message 353.  

Would that even be of value? Unless they are sending the 'meat' of the task output back in the report, all the the really useful science stuff would be gone already in the output file upload.

All that meat is sent back with the report. It's all logged in client_state.xml
All the project got thus far was the data that the people crunched and uploaded, the output file. Not everything else you see in stdout_err.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 354 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Heavy Metal Dungeon Keeper

Send message
Joined: 2 May 09
Posts: 22
Credit: 81,216
RAC: 0
Message 355 - Posted: 14 May 2009, 16:11:41 UTC
Last modified: 14 May 2009, 16:20:34 UTC

If I remember correctly I had around 6.5K of pending credit at one time from the previous batch to the one I cannot report, looking at BOINCstats on 12 May, my total credit jumped 8300.

Did Jack do something to validate all the wus outstanding from the previous batch betweeen BOINCstats main updates on 11th and 12th of May also looking at DD@H stats, there was a big jump in credit awarded on the 12th May by the project. overall.
ID: 355 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Steven Pletsch
Avatar

Send message
Joined: 1 May 09
Posts: 14
Credit: 39,153
RAC: 0
Message 356 - Posted: 14 May 2009, 16:48:15 UTC

Just a stab in the dark... but..

Are you using the same PHP installation for the BOINC site and the Drupal site ?

I have seen where there is a "feature" in Drupal, where it silently limit the file_upload_max_size to half the post_max_size. If you are using the same PHP installation, it might be worth looking at.

http://drupal.org/node/104220

ID: 356 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Tim Turner
Avatar

Send message
Joined: 1 May 09
Posts: 570
Credit: 184,322
RAC: 0
Message 357 - Posted: 14 May 2009, 17:05:53 UTC - in response to Message 356.  
Last modified: 14 May 2009, 17:06:15 UTC

ok, i admit that sometimes my idea's suck.

but, saving(backing it up) the file isn't a bad idea. you never know.
ID: 357 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 358 - Posted: 14 May 2009, 17:20:53 UTC

We're getting help from the big guns now. Thus far it looks like a BOINC scheduler bug. If SciTechGrid can send me the file I requested in PM, it would go a bit quicker. ;-)

Hurry it up, man! lol
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 358 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 359 - Posted: 14 May 2009, 17:45:39 UTC - in response to Message 358.  

With thanks to everyone who sent me their sched_request*.xml file. I think the developers have enough for now. :-)
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 359 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile KSMarksPsych

Send message
Joined: 22 Apr 09
Posts: 5
Credit: 2,010
RAC: 0
Message 369 - Posted: 14 May 2009, 21:01:41 UTC

Mine all reported! With many thanks!
ID: 369 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 374 - Posted: 14 May 2009, 21:41:14 UTC

In case you missed it all, the reporting problem is fixed. You can report all your outstanding work. With thanks to David Anderson for the help and the fix in the back-end code.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 374 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Bok

Send message
Joined: 21 Apr 09
Posts: 5
Credit: 261,336
RAC: 1
Message 378 - Posted: 14 May 2009, 23:37:35 UTC

I'm still getting the HTTP internal server error..

Should I be stopping the client and restarting ?
ID: 378 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 379 - Posted: 14 May 2009, 23:55:54 UTC - in response to Message 378.  

ok, so if that does not work, try resetting. Still having problems? send me your

sched_request_boinc.drugdiscoveryathome.com.xml
ID: 379 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile KSMarksPsych

Send message
Joined: 22 Apr 09
Posts: 5
Credit: 2,010
RAC: 0
Message 388 - Posted: 15 May 2009, 9:29:31 UTC - in response to Message 379.  

ok, so if that does not work, try resetting. Still having problems? send me your

sched_request_boinc.drugdiscoveryathome.com.xml


If you reset the project, you lose all the work that's on your machine, even if it's ready to report.
ID: 388 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 389 - Posted: 15 May 2009, 10:48:44 UTC
Last modified: 15 May 2009, 14:27:21 UTC

/me continues writing the How-To on manually reporting all that work. I need a victim... um test-person though who can test it for me before I release the steps to the wild. PM me if you're interested.

Bok PM'ed me and is willing to play guinea pig. Keep your fingers crossed all. :-)
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 389 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 399 - Posted: 15 May 2009, 18:57:53 UTC - in response to Message 398.  
Last modified: 15 May 2009, 19:07:01 UTC

No, it wasn't.

I have written the How-To, have had Bok guinea pig through it. He didn't lose any of his work, so that's good. But on the other hand, he didn't even manage to report 50 tasks at a time while using the How-To. So something else is really wrong here.

Edit: it seems that if you're stuck in this situation that it could be even worse, that you need a detach/reattach. Before that time though, can people who find they still can't report their work please tell us about it?
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 399 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 405 - Posted: 15 May 2009, 22:58:08 UTC

It is going to take a little longer. We still have a bug in the scheduler that drops connections or crashes due to running out of memory.

In the mean time, a crude way of doing things is to reset. Drugdiscovery has resend lost results on, so you should be resent the tasks you just lost and have to do them again. Then you can try to report them in smaller batches.

But otherwise wait until we're done fixing the scheduler.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 405 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Tim Turner
Avatar

Send message
Joined: 1 May 09
Posts: 570
Credit: 184,322
RAC: 0
Message 802 - Posted: 15 Jul 2009, 21:36:41 UTC - in response to Message 409.  
Last modified: 15 Jul 2009, 21:38:24 UTC

seems this is on a comeback tour with this batch... bringing it back up..

as Steven is having a problem..

also, adding, a good way to prevent this is to set your cache to 1 day or less and have your computer connect every .1 days....
ID: 802 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : Scheduler request failed: HTTP internal server error


©2017 All rights reserved | Design by Digital BioPharm Ltd