Project Status 2010

Message boards : Number crunching : Project Status 2010

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 6 · Next

AuthorMessage
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 91 - Posted: 3 May 2009, 3:35:07 UTC - in response to Message 82.  
Last modified: 3 May 2009, 3:35:39 UTC

What happened to DD@H!!! Its been down for the past 6 hours!!! Its the end of the World! lol

Ok, I felt an urge to backup the server. I thought it would not be a problem. I've done it before ok? This time it was not as easy as I thought.

1st our Virtual Private Server wedge the snapshot process when it was running live. It was stealing resources from the rest of the server. My hosting provider had to put a stop to that. Then we tried backup again. It caused the same problem even when I stopped the daemons.

Ok so I really want a backup. I do my snapshot while it is down. This takes 1.5 hours. Much more than I thought it would. I watch a movie for the first time, "Slum Dog Millionaire" I would say one of the greatest movies of all time, and I think it was influenced by the works of Charles Dickens. Afterwords, I see it is done backing up.
ID: 91 · Rating: 0 · rate: Rate + / Rate - Report as offensive
zombie67 [MM]
Volunteer tester
Avatar

Send message
Joined: 25 Apr 09
Posts: 58
Credit: 1,785,257
RAC: 224
Message 92 - Posted: 3 May 2009, 3:43:20 UTC - in response to Message 91.  

ID: 92 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 112 - Posted: 5 May 2009, 22:44:00 UTC - in response to Message 92.  

We have done some integration with Drupal. Just used some iframes for now. Not the best type of integration but do I really want to write a bunch of BOINC modules when the BOINC Developers already have that project?
ID: 112 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 120 - Posted: 6 May 2009, 19:07:36 UTC - in response to Message 20.  

We could use some more hosts. Team leaders, can you contact me for invite codes?
ID: 120 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 124 - Posted: 7 May 2009, 3:16:48 UTC - in response to Message 120.  

Watch out for some of those GROMACS workunits! they have some gremlins in them. If you get them running for a while look at the standard error. If they keep repeating the same output, kill them.
ID: 124 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 129 - Posted: 7 May 2009, 20:31:36 UTC
Last modified: 8 May 2009, 16:49:21 UTC

DD@H is now available on BOINStats. Mundayweb will follow when Neil is back from his business trip. BOINC Synergy is still down, but I'll email Zain about having us added to their stats anyway.

By the way, it looks like perhaps the stats exported are broken as we're showing zero Active users, teams and countries. :-(
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 129 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 151 - Posted: 8 May 2009, 11:51:00 UTC - in response to Message 129.  

Now we have several newcomers to DD@H. Welcome. For the past week the project has gone through some turbulence. The server backups were problematic and we migrated to a new host. Our workunit queue is large. There are some download errors that we need to address. Once this current batch of workunits completes, I want to clean up the server, upgrade the project. Then load the input files onto the server. If we have the input already loaded, and store them with 7z format and use the nodelete option. We may find it easier to generate these workunits. I have not done things this way before, but it seems like the best way to manage disk space. Right now we have lots of redundant data stored on the server.

As far as our applications go, they are less than ideal but they work at least 90% of the time, maybe linux 64-bit has a lot more trouble. They consume way too much disk space and there is no checkpointing. This requires quite a bit of programming to include the boinc api in both C++ code but also Python scripts. I've attempted to add the boinc api functions to Autodock about 2 years ago. I gave up on it. If anyone here wants to give it a try, they have the code available from The Scripps Research Institute. For me, checkpointing is a lower priority than actually getting useful data from our analysis.
ID: 151 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 166 - Posted: 9 May 2009, 14:25:57 UTC - in response to Message 151.  

I know we have a couple new people. Hang in there. I want to let our current pending results clear before I start with a fresh batch. This time we will apply some more refined approaches to filtering the input. For instance, check if the file is empty :-) |o|

I would like to actually prep these ligand files and store the pdbqt files on our server so we can skip this step. For our database of molecules, we will have to convert some 10,000 molecular structures. These are very quick workunits. They should take less than 5 minutes. One might even argue this could be done on one computer. At least it gives us a bit more experience in workunit management. I want to also test the 7zip compression in these workunits.

So wait a few days until the queue clears on our project status page. Then expect some workunits.

Best Regards,

Jack
ID: 166 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 178 - Posted: 10 May 2009, 14:06:38 UTC - in response to Message 129.  

DD@H is now available in BAM!
You will need to have an account here first, then choose the Also show projects that do not accept new members option to see the project.

Mundayweb has added DD@H to its statistics. BOINC Synergy is back up but hasn't added it yet. Someone else may want to prod Zain as well.
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 178 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 190 - Posted: 11 May 2009, 12:58:53 UTC - in response to Message 178.  

I just submitted the entire library of PLATED compounds for conversion from mol2 to pdbqt format. These workunits should take less than a minute but its an enormous number of jobs. I figure this conversion would be a good test and produce useful data. If we keep these results handy, we won't have to do the prepare_ligand step for this library of molecules.
ID: 190 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 200 - Posted: 11 May 2009, 14:14:00 UTC - in response to Message 190.  

I'm wondering if I'm being too ambitious today over 600,000 workunits may take too long to process.
ID: 200 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 284 - Posted: 13 May 2009, 15:05:39 UTC - in response to Message 200.  
Last modified: 13 May 2009, 15:11:14 UTC

One thing to consider, if we ever adopt the parallel API.

How well we could do analysis that requires frequent communication with a server. There is a lot of debate whether this API will ever have an advantage. I will say that one limitation with running molecular dynamics simulations is the amount of memory they consume as they get bigger.

I hope Jaspal's project will eventually allow us to divide the analysis of a certain number of atoms in a system to different hosts. Imagine simulating a system of 1 million atoms among 10 thousand computers. The simulation would get nowhere on one desktop.

Jaspal Subhlok: Inter-task communication on volatile nodes. Slides, Video
http://boinc.berkeley.edu/trac/attachment/wiki/WS08/WorkshopProceedings/jaspal_subhlok_sync.pdf
http://video.google.com/videoplay?docid=6560663072045801424&hl=en
ID: 284 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 299 - Posted: 14 May 2009, 0:43:06 UTC - in response to Message 284.  
Last modified: 14 May 2009, 0:50:31 UTC

Hey there guys. I was generating some new work for the FZD2 receptors. Some of these did not have the right file open names in the workunit templates. I fixed that as soon as I noticed. I then waited for some sucessful workunits. I did a loop of 100 workunits. There are roughly 10,000 ligands in this database. The workunits should take 1 hour. We need three successful runs to validate. 3 million hours divided by about 336 hosts...10,000 cpu hours. This could take us over a year with this many hosts :-(

So it may be time to really ramp up and call for more volunteers!!!
ID: 299 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Ageless
Avatar

Send message
Joined: 11 Apr 09
Posts: 172
Credit: 7,631
RAC: 0
Message 329 - Posted: 14 May 2009, 10:54:42 UTC
Last modified: 14 May 2009, 11:39:43 UTC

I know, this is going completely against what Jack said above, but things have changed overnight.

After some deliberation between Andrey, Jack and me, we have decided that we have enough testers for the moment and as such have stopped giving out the invitation code. First we want to solve some of the problems that we have around here, before we give out the new invitation code.

There's no need to keep requesting the code for the moment. Sorry for you who have missed out for now, welcome to all new that managed to still sneak in. :-)

(Team Founders whose accounts are already made, just waiting to be activated, can still use the pointer in this thread to activate their account).
Jord

'Cause you seem like an orchard of mines, Just take one step at a time.
ID: 329 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 370 - Posted: 14 May 2009, 21:30:48 UTC - in response to Message 329.  

Kathryn was the first to report after I implemented the new update. It should be reporting now.
ID: 370 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 382 - Posted: 15 May 2009, 2:09:53 UTC - in response to Message 370.  

Account creation
Andrew, our project director would like to suspend account creation until we resolve all outstanding issues and are ready for production analysis.
ID: 382 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 390 - Posted: 15 May 2009, 11:56:10 UTC - in response to Message 382.  

Uploads
Apparently we are not out of the woods yet. Some results did get uploaded. I think Agless is preping some instructions for how to troubleshoot this again. We are also waiting for a response from the BOINC developers.
ID: 390 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 414 - Posted: 16 May 2009, 1:42:07 UTC - in response to Message 390.  

We got some tremendous help from our friends at Berkeley. We updated the server software. Still some people may have trouble. Please send us your sched_request_boinc.drugdiscoveryathome.com.xml
ID: 414 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 423 - Posted: 17 May 2009, 5:27:36 UTC - in response to Message 414.  

please update the project if you have results to report. I need to hear from you if you continue to have issues reporting results. I think one of our best volunteer developers helped me patch the problem with our scheduler.
ID: 423 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Jack Shultz
Avatar

Send message
Joined: 10 Apr 09
Posts: 503
Credit: 120,150
RAC: 0
Message 433 - Posted: 19 May 2009, 10:47:22 UTC - in response to Message 423.  

I've had enough. I'm going to cancel all workunits. Abort anything you have.
ID: 433 · Rating: 0 · rate: Rate + / Rate - Report as offensive
1 · 2 · 3 · 4 . . . 6 · Next

Message boards : Number crunching : Project Status 2010


©2017 All rights reserved | Design by Digital BioPharm Ltd