Treats and Tricks, or use SimGrid - USS SimGrid
Contents
1. finalize break end task execute MSG debug Slave mailbox done executing task task name end MSG info I m done see you end end Setting up your experiment MSG createEnvironment platform xm1 MSG deployApplication deploy xm1 MSG run puts Simulation time MSG getClock to_s MSG exit Some more polishing is needed gt Exceptions on timeout host failure and so on Martin Quinson Treats and Tricks or use SimGrid The bindings 23 26 MSG bindings performance What about performance loss for Java Warning these values are 2 years old gt Small platforms ok gt Larger ones not quite workers 100 500 1 000 5 000 10 000 tasks 1 000 native 16 19 21 42 0 74 java Al 59 94 76 27 10 000 native 48 52 54 83 11 java 16 1 9 238 13 40 100 000 native 3 7 3 8 40 44 45 java 14 13 15 29 77 1 000 000 native 36 37 38 41 40 java 121 130 134 163 200 What about the others gt It s too recent didn t had time to rerun the full benchs gt Very preliminary results for Master slaves 10 slaves 200 000 tasks Martin Quinson Treats and Tricks or use SimGrid C native 7s gt That s improvable Lua 10 5s gt It s garbage collected Ruby 45s gt User stack is dynamic in lua amp ruby Java 47s better scalability The bindings 242. messageSize public class FinalizeTask extends simgrid msg Task public FinalizeTask super finalize 0 0 public class Worker extends simgrid msg Process public void main String args throws TransferFailureException HostFailureException TimeoutException TaskCancelledException String id args 0 while true Task t Task receive worker id if t instanceof FinalizeTask break BasicTask task BasicTask t Msg info Processing task getName task execute Msg info task getName gt done Msg info Received Finalize I m done See you ye 18 26 Martin Quinson Treats and Tricks or use SimGrid The bindings Master slaves in Java 2 2 import simgrid msg public class Master extends simgrid msg Process public void main String args throws MsgException int numberOfTasks Integer value0f args 0 intValue double taskComputeSize Double value0f args 1 doubleValue double taskCommunicateSize Double value0f args 2 doubleValue int workerCount Integer value0f args 3 intValue Msg info Got workerCount workers and numberOfTasks tasks for int i 0 i lt number0fTasks i BasicTask task new BasicTask Task_ i taskComputeSize taskCommunicateSize task send worker i workerCount Msg info Send completed for the task task getName on the mailb
3. 26 XBT from 10 000 feets C is a basic language we reinvented the wheel for you Logging support Log4C XBT_LOG_NEW_DEFAULT_CATEGORY test my own little channel XBT_LOG_NEW_SUBCATEGORY details test Another channel INFO1 Value d variable CDEBUG3 details blah d f 4d x y z xbt_ex_t e TRY block CATCH e block DO NOT RETURN FROM THERE Exception support J ___ Debugging your code gt Ctrl C once see processes status gt Press it twice in 5s kill simulator xbt_backtrace_display_current Backtrace displayed in thread 0x90961c0 gt In master at masterslave_mailbox c 35 gt In 0x4a69ba5 Martin Quinson Treats and Tricks or use SimGrid Advanced data structures gt Hash tables Perl s ones gt Dynamic arrays FIFOs gt SWAG don t use Graphs String functions gt bprintf malloc ing sprintf gt trim split subst diff gt string buffers Threading support gt Portable wrappers Lin Win Mac Sim gt Synchro mutex conds semaphores Other Mallocators Configuration support Unit testing check src testall Integration tests tesh testing shell v vvyv Surviving in C 25 26 Conclusion Finding the documentation Martin Quinson Treats and Tricks or use SimGrid Conclusion 1 26 26 Conclusion Finding the documentation User manuals are for wimps gt Real Men read some slide
4. convinient for MPI you d have to merge them gt Simply pass NULL to MSG_action_trace_run gt Pass the trace file to use as argument to each process in deployment lt xml version 1 0 gt lt DOCTYPE platform SYSTEM simgrid dtd gt lt platform version 2 gt lt process host Tremblay function toto gt lt argument value actions_toto txt gt lt process gt lt process host Jupiter function tutu gt lt argument value actions_tutu txt gt lt process gt lt platform gt Action Semantic gt This mecanism is completely agnostic attach the meaning you want to events gt In examples actions action c we have pre written event functions for gt Basics send recv sleep compute gt MPl specific isend irecv wait barrier reduce bcast allReduce Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 6 26 SimGrid is not a Simulator That s a Generic Simulation Framework Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 4 7 26 gt Outline Wannabe User Manual Configuring and Installing Configuring your simulators Trace Replay Some Do not n advices Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual l 8 26 Do not mix results between releases Main issue The events order does change gt Models don t change only the order of events occuring at the same time gt It may will change your appli
5. mailbox number to use gt lt process host Jupiter function worker gt lt argument value 0 gt lt process gt lt process host Fafard function worker gt lt argument value 1 gt lt process gt lt process host Ginette function worker gt lt argument value 2 gt lt process gt lt platform gt Thanks to mailboxes the master don t have to know where the slaves live nor the contrary Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 14 26 The MSG master workers example the main Putting things together int main int argc char argv MSG_global_init karge argv Declare all existing agent binding their name to their function MSG_function_register master amp master MSG_function_register worker amp worker Load a platform instance MSG_create_environment my_platform xml Load a deployment file MSG_launch_application my_deployment xm1 Launch the simulation until its end MSG_main INFO1 Simulation took g seconds MSG_get_clock Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 15 26 Outline The bindings Java Lua Ruby Martin Quinson Treats and Tricks or use SimGrid The bindings 16 26 The bindings Some people don t like coding in C gt We have some Java bindings since 2008 at least gt We have some Lua bindings gt We have some Ruby bindings Why these languag
6. Treats and Tricks or use SimGrid Martin Quinson SimGrid User Days 2010 Cargese SIM BRID Outline Wannabe User Manual Configuring and Installing Configuring your simulators Trace Replay Some Do not n advices e The bindings Java Lua Ruby Surviving in C Conclusion Martin Quinson Treats and Tricks or use SimGrid 2 26 How to install the tool In release 3 3 4 and before gt Grab it https gforge inria fr project showfiles php group_id 12 gt configure amp amp make amp amp sudo make install In release 3 4 and afterward gt Grab it cmake amp amp make amp amp sudo make install simgrid Get it from SVN gt svn checkout svn scm gforge inria fr svn simgrid simgrid trunk simgrid gt cd simgrid gt cmake amp amp make amp amp sudo make install simgrid From autotools to cmake gt So yeah we moved from autoconf to cmake for release 3 4 gt It offers a better user interface with ccmake demo gt If allows for software quality dashboards demo Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual l 3 26 Configuring your simulators Every simulator using SimGrid accepts a set of options help get some help demo help models long help on models 3 4 only demo log configure the verbosity cfg change some settings Note SMPI specific settings are only visible in SMPI simulators The log argument gt It s similar to L
7. cation s behavior if gt In a master slaves sending tasks in a round robin gt In a scheduling algorithm considering the ready tasks in order gt In almost every application actually Why that gt We improve the data structure used for future event set gt We sort the events to not traverse the ones which cannot be done yet gt We do lazy evaluation Side note gt When this happens there is a big fat warning in the ChangeLog gt you should read it anyway don t wait for the next SUD to get infos Optimistic note gt The MSG interface not changed since 2002 backward compatibility gt At least with MSG_USE_DEPRECATED Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 9 26 Do not use the lt cluster gt tag But you got it already right gt We know the issue will fix it at some point Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 10 26 Stop using MSG _task_put MSG task_get Now Solving the rendez vous issues in task exchanges gt MSG have a strange interaface somehow mimicking BSD sockets ports gt You send to host port as in BSD gt but 2 people could send to the same which is somehow ok gt and 2 people could get from the same which is annoying gt Now we have a much cleaner interface based on mailboxes Mailboxes gt Mailboxes are represented by a string whatever you would like gt You send stuff to a mailbox you receive stuff fr
8. es gt Every potential intern knows Java I guess gt Lucas the office next to mine is a very effective Ruby lobbyist gt Lua is said to allow very efficient bindings with C Will you add my favorite language gt We could but it s rather time consuming threading mess at least gt I m not willing to start a collection here Medhi s time is limited Patch welcome and wish you good luck we see it in next SUD Martin Quinson Treats and Tricks or use SimGrid The bindings l 17 26 The bindings Some people don t like coding in C gt We have some Java bindings since 2008 at least gt We have some Lua bindings since March 23 2010 gt We have some Ruby bindings since April 7 2010 Why these languages gt Every potential intern knows Java I guess gt Lucas the office next to mine is a very effective Ruby lobbyist gt Lua is said to allow very efficient bindings with C Will you add my favorite language gt We could but it s rather time consuming threading mess at least gt I m not willing to start a collection here Medhi s time is limited Patch welcome and wish you good luck we see it in next SUD Martin Quinson Treats and Tricks or use SimGrid The bindings l 17 26 Master slaves in Java 1 2 import simgrid msg public class BasicTask extends simgrid msg Task public BasicTask String name double computeDuration double messageSize super name computeDuration
9. ize then simgrid info Slave my_mailbox got finalize msg break end simgrid Task execute tk end simgrid info Slave my_mailbox I m Done See You end Setting up your experiment require simgrid simgrid platform my_platform xml simgrid application my_deployment xm1 simgrid run simgrid info Simulation s over See you simgrid clean Martin Quinson Treats and Tricks or use SimGrid The bindings 21 26 Master slaves in Ruby 1 2 Some mandatory headers require simgrid include MSG The master class Master lt MSG Process def main args number0fTask Integer args 0 taskComputeSize Float args 1 taskCommunicationSize Float args 2 slaveCount Integer args 3 for i in O numberOfTask 1 task Task new Task_ i to_s taskComputeSize taskCommunicationSize mailbox slave i slaveCount to_s MSG info Master Sending task name to mailbox task send mailbox MSG info Master Done Sending task name to mailbox end for i in 0 slaveCount 1 mailbox slave i to_s finalize_task Task new finalize 0 0 finalize_task send mailbox end end end Martin Quinson Treats and Tricks or use SimGrid The bindings 22 26 Master slaves in Ruby 2 2 The slave class Slave lt MSG Process def main args mailbox slave args 0 while true task Task receive mailbox if task name
10. og4J but in C gt You can increase the amount of output for some specific parts of SimGrid gt Example See everything by using log root thres debug demo gt List of all existing channels doc html group_XBT_log_ cats html Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual l 4 26 Trace Replay Separate your applicative workload C code static void action_blah xbt_dynar_t parameters static void action_blih xbt_dynar_t parameters static void action_bluh xbt_dynar_t parameters int main int argc char argv MSG_global_init amp argc argv MSG_create_environment argv 1 MSG_launch_application argv 2 No need to register functions as usual actions started anyway MSG_action_register blah blah MSG_action_register blih blih MSG_action_register bluh bluh MSG_action_trace_run argv 3 The trace file to run Deployment Trace file tutu blah toto 1e10 toto blih tutu tutu bluh 12 toto blah 12 lt xml version 1 0 gt lt DOCTYPE platform SYSTEM simgrid dtd gt lt platform version 2 gt lt process host Tremblay function toto gt lt process host Jupiter function tutu gt lt process host Fafard function tata gt lt platform gt Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 5 26 Trace Replay 2 2 Separating the trace of each process gt Because it s sometimes more
11. om a mailbox gt Where in network the sender and receiver are do not matter for rendez vous gt The communication timings of course take their locations into account gt This makes the user code ways easier v know that the examples were not all updated yet sorry gt Read examples msg masterslave masterslave_mailbox c ignore others Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 11 26 The cleaner Master Workers in MSG with mailboxes The master has a large number of tasks to dispatch to its workers for execution int master int argc char argv int number_of_tasks atoi argv 1 double task_comp_size atof argv 2 double task_comm_size atof argv 3 int workers_count atoi argv 4 char mailbox 80 char buff 64 int i Dispatching dumb round robin algorithm for i 0 i lt number_of_tasks i sprintf buff Task_Z d i task MSG_task_create sprintf_buffer task_comp_size task_comm_size NULL sprintf mailbox worker d i workers_count INFO2 Sending s to mailbox s task gt name mailbox MSG_task_send task mailbox Send finalization message to workers INFOO Al1 tasks dispatched Let s stop workers for i 0 i lt workers_count i sprintf mailbox slave ld i slaves_count MSG_task_send MSG_task_create finalize 0 0 0 mailbox INFOO Goodbye now return 0 Martin Quinson Treats and Tricks
12. or use SimGrid Wannabe User Manual 12 26 The MSG master workers example the worker int worker int argc char argv m_task_t task int errcode int id atoi argv 1 char mailbox 80 sprintf mailbox worker d id while 1 errcode MSG_task_receive amp task mailbox xbt_assertO errcode MSG_OK MSG_task_get failed if strcmp MSG_task_get_name task finalize MSG_task_destroy task break INFO1 Processing s MSG_task_get_name task MSG_task_execute task INFO1 s done MSG_task_get_name task MSG_task_destroy task INFOO I m done See you return 0 Martin Quinson Treats and Tricks or use SimGrid Wannabe User Manual 13 26 The MSG master workers example deployment file Specifying which agent must be run on which host and with which arguments XML deployment file lt xml version 1 0 gt lt DOCTYPE platform SYSTEM surfxml dtd gt lt platform version 2 gt lt The master process with some arguments gt lt process host Tremblay function master gt lt argument value 6 gt lt Number of tasks gt lt argument value 50000000 gt lt Computation size of tasks gt lt argument value 1000000 gt lt Communication size of tasks gt lt argument value 3 gt lt Number of workers gt lt process gt lt The worker process argument
13. ox worker i workerCount Msg info Goodbye now The rest of the story gt No need to write the glue thanks to Java introspection gt The XML files are exactly the same beware of capitalization for deployment gt Output very similar too Martin Quinson Treats and Tricks or use SimGrid The bindings 19 26 Master slaves in Lua 1 2 function Master nb_task arg 1 comp_size arg 2 comm_size arg 3 slave_count arg 4 Dispatch the tasks for i 1 nb_task do tk simgrid Task new Task i comp_size comm_size alias slave i4slave_count simgrid info Master sending simgrid Task name tk To alias simgrid Task send tk alias simgrid info Master done sending simgrid Task name tk To alias end Sending Finalize Message To Others for i 0 slave_count 1 do alias slave i simgrid info Master sending finalize to alias finalize simgrid Task new finalize comp_size comm_size simgrid Task send finalize alias end end Some more polishing is needed gt We d prefer tk send alias instead of simgrid Task send tk alias Martin Quinson Treats and Tricks or use SimGrid The bindings 20 26 Master slaves in Lua 2 2 The slave function Slave local my_mailbox slave arg 1 while true do local tk simgrid Task recv my_mailbox if simgrid Task name tk final
14. s cause they are more concise gt They read the examples pick one modify it to fit their needs gt They may read 2 or 5 of the reference guide to check the syntax gt In doubt they just check the source code Martin Quinson Treats and Tricks or use SimGrid Conclusion l 26 26 Conclusion Finding the documentation User manuals are for wimps gt Real Men read some slides cause they are more concise gt They read the examples pick one modify it to fit their needs gt They may read 2 or 5 of the reference guide to check the syntax gt In doubt they just check the source code lusers don t read the manual either gt Proof that s why the RTFM expression were coined out gt Instead they always ask same questions to lists and get pointed to the FAQ Martin Quinson Treats and Tricks or use SimGrid Conclusion l 26 26 Conclusion Finding the documentation User manuals are for wimps gt Real Men read some slides cause they are more concise gt They read the examples pick one modify it to fit their needs gt They may read 2 or 5 of the reference guide to check the syntax gt In doubt they just check the source code lusers don t read the manual either gt Proof that s why the RTFM expression were coined out gt Instead they always ask same questions to lists and get pointed to the FAQ So where is all SimGrid documentation gt The SimGrid tutorial is a 200 slides presentation motivation model
15. s example of use internals gt Almost all features of UAPI are demoed in an example coverage testing gt The reference guide contains a lot in introduction sections about XBT gt The FAQ contains a lot too installing visu XML exotic features gt The code is LGPL anyway Martin Quinson Treats and Tricks or use SimGrid Conclusion l 26 26
Download Pdf Manuals
Related Search