Skip to main content

Bash - parallel operations

Today I would like to present useful tool for every Linux developer - bash parallel mode. Bash parallel mode allows to invoke multiple bash tasks (ex. functions, linux applications etc.) in parallel, which could save time of processing them. What's is more, usage of bash parallel mode is very simple and does not require lot of work. Let's explain it based on the examples. We are starting with bash script working in seqence mode (one function invoked after previous one). We defind function task() which is later invoked in for loop 10 times. Invoking of each instance of function takes SLEEP_TIME=2 seconds. So script invokation takes 10x2s == 20 seconds. Now, lets change above script in order to work in parallel mode: Please note few small, but important changes between above scripts: <
  • invokation of function task has '&' character at the end. It means that function will be invoked in background and will not block main thread of script.
  • function is invoked before end of program. It is very important function, as it allows main thread of script to wait until all of its background jobs will be finished. Otherwise task invoked in background would became zombie process
And that's it. Now all invokations of tasks are invoked in parallel. After invoking above script you will see that it takes about 2seconds to invoke that script (it is around of time of one task() function invokation) As you can see invoking bash tasks in parallel is very streighforward, but it can save time of processing. However we still need to remember that working multiple tasks which share memory each other, may cause the same problems as multithreading in C/C++ programming (data race conditions (described here: Race conditions) and deadlocks). So you should be aware and take care of those problems when working in absh parallel mode.

 
Labels:

Comments

Post a Comment

Popular posts from this blog

Blog's new layout

As you noticed this blog has new layout from today. I hope you like it. I think new layout looks better and more modern than previous one. Please, write you opinion about new layout in comments. If you have some ideas how to make this blog better, all ideas are welcomed. Enjoy new layout and blog articles.
3 comments
Read more

QT - foreach algoriithm with const references performance improvement

Today I would like to show you optimal way of using foreach QT algorithm . I will show you why we should pass elements of foreach algorithm by const reference instead of passing them by value. Let me explain it on the below example: Output of this example is: In point I we are creating 3 objects of MyClass class and push them to myClasses QList element. In point II we are using QT foreach algorithm to invoke getValue() method for each object from myClasses list. As you can see on output text for that part of code we are invoking copy constructor before and destructor after invoking getValue() function. It is because we are passing each myClasses list element to foreach algorithm by value. Therefore we are copying that element at the beginning of foreach loop step and removing them (destructing) at the end. This is inefficient solution, especially when class of object being copied is big. It decreases performance. of our application. Solution for that i...
1 comment
Read more

Advanced C++ - Compiler generated functions

When we define some class in our C++ code, compiler is generating some important functions for our class (unless we define it explicitly). Those functions (sometimes called Compiled Generated Functions ) are:  default constructor   destructor   copy constructor   copy assignment operator  During compilation, compiler knows our code and classes usage, so when we are using one of above functions implicitly, it generate that function implicitly for us in the class body. For better explenation, let's see the example below: In point (I) we are defining class TestClass . This class seems to be empty. However we are implicitly using following function in main class:  point III - we are implicitly using defualt constructor in order to create instance of TestClass - default constructor of TestClass is implicitly generated by compiler   point IV - we are using copy constructor of TestClass in order to copy instance to...
2 comments
Read more