Bash - parallel operations

Today I would like to present useful tool for every Linux developer - bash parallel mode. Bash parallel mode allows to invoke multiple bash tasks (ex. functions, linux applications etc.) in parallel, which could save time of processing them. What's is more, usage of bash parallel mode is very simple and does not require lot of work. Let's explain it based on the examples. We are starting with bash script working in seqence mode (one function invoked after previous one). We defind function task() which is later invoked in for loop 10 times. Invoking of each instance of function takes SLEEP_TIME=2 seconds. So script invokation takes 10x2s == 20 seconds. Now, lets change above script in order to work in parallel mode: Please note few small, but important changes between above scripts: <

invokation of function task has '&' character at the end. It means that function will be invoked in background and will not block main thread of script.
function is invoked before end of program. It is very important function, as it allows main thread of script to wait until all of its background jobs will be finished. Otherwise task invoked in background would became zombie process

And that's it. Now all invokations of tasks are invoked in parallel. After invoking above script you will see that it takes about 2seconds to invoke that script (it is around of time of one task() function invokation) As you can see invoking bash tasks in parallel is very streighforward, but it can save time of processing. However we still need to remember that working multiple tasks which share memory each other, may cause the same problems as multithreading in C/C++ programming (data race conditions (described here: Race conditions) and deadlocks). So you should be aware and take care of those problems when working in absh parallel mode.

Comments

GDB Automatic Deadlock Detector

Have you ever had a problem with detection deadlock between threads in your C/C++ application? Would you like to do that automatically? Try this GDB Automatic Deadlock Detector from my github: GDB Automatic Deadlock Detector Picture source: http://csunplugged.org/wp-content/uploads/2015/03/deadlock.jpg1286488735

STL - count and count_if algorithms

One of the basic and most useful STL algorithms is algorithm which can be used to count number of elements within selected container according to specified criteria. In order to do that we can use std::count or std::count_if algorithm. std::count (firstElementIterator, lastElementIterator, elementForSearch) - is function which will go through container using firstElementIterator and lastElementIterator and return number of container elements which value is equal elementForSearch std::count_if (firstElementIterator, lastElementIterator, UnaryPredicateFunction) - is function which examine range from firstElementIterator to lastElementIterator and return number of container elements which fulfill UnaryPredicateFunction criteria. UnaryPredicateFunction is function having following signature: bool functionName(const Type& a) . So, count_if returns number of elements where UnaryPredicateFunction returns true for. For better understanding let's take a

Advanced C++ - Stack unwinding

Stack unwinding is normally a concept of removing function entries from call stack (also known as Execution stack, Control stack, Function stack or Run-time stack). Call Stack is a stack data structure that stores active functions' addresses and helps in supporting function call/return mechanism. Every time when a function is called, an entry is made into Call stack which contains the return address of the calling function where the control needs to return after the execution of called function. This entry is called by various names like stack frame , activation frame or activation record. With respect to exception handling , stack Unwinding is a process of linearly searching function call stack to reach exception handler. When an exception occurs, if it is not handled in current function where it is thrown, the function Call Stack is unwound until the control reaches try block and then passes to catch block at the end of try block to handle exception. Also, in this proc

Developer Adventure

Search This Blog