Monday, March 31, 2014

Improving Performance - Part 3 - Parallel processing

Running at the same time instead of in sequence can sometimes be one of the biggest gains in performance you can make to your job.

Imagine you have 3 data flows of 10 minutes each. If you run them sequentially it will take 30 minutes, but if you run them at the same time it would only take 10 minutes in total.... provided of course that your job server has sufficient resources to run all 3 at the same time.

Now you can either go about working out on your own which data flows can be run sequentially or in parallel, or you can let Data Services work it all out for you.

To get Data Services to do the hard work for you, read my blog post on developing Data Services jobs using components and how the execute only once option helps you out.


No comments:

Post a Comment