Simple parallel processing in D with std.parallelism

By Vasudev Ram

Phobos image attribution

The D language has a module for parallel processing support. It is in Phobos, D's standard library (guess why it is called Phobos :). [1]
The module is called std.parallelism.

From the Phobos page:

[ Generally, the std namespace is used for the main modules in the Phobos standard library. The etc namespace is used for external C/C++ library bindings. The core namespace is used for low-level D runtime functions. ]

Here are a couple of simple D programs that together show the speedup that can be obtained by using the parallelism module. Note that this is for tasks that involve the CPU, not I/O.

The first one, student_work_sequential.d, does not use the std.parallelism module, so it does 4 tasks sequentially, i.e. one after another.

The second one, student_work_parallel.d, does use it. It does the same 4 tasks as the first. But it uses the convenience function called parallel from std.parallelism, to run the 4 tasks in parallel.
I timed the results of running each program a few times, and the parallel one was consistently over 3 times faster than the sequential one.

Each of the D programs below can be compiled with the command:

dmd program_name.d

Here is the code for student_work_sequential.d:

/*
student_work_sequential.d
Author: Vasudev Ram
Copyright 2016 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: http://jugad2.blogspot.com
Product Store: https://gumroad.com/vasudevram
Twitter: https://mobile.twitter.com/vasudevram
*/

import std.stdio;
import core.thread;

struct Student {
    int number;
    void doSlowOperation() {
        writefln("The work of student %s has begun", number);
        // Wait for a while to simulate a long-lasting operation
        Thread.sleep(1.seconds);
        writefln("The work of student %s has ended", number);
    }
}

void main() {
    auto students =
        [ Student(1), Student(2), Student(3), Student(4) ];
    foreach (student; students) {
        student.doSlowOperation();
    }
}

Here is the output from running it, under the control of a command timing program written in Python:

$ python c:\util\time_command.py student_work_sequential
The work of student 1 has begun
The work of student 1 has ended
The work of student 2 has begun
The work of student 2 has ended
The work of student 3 has begun
The work of student 3 has ended
The work of student 4 has begun
The work of student 4 has ended
Command: student_work_sequential
Time taken: 4.07 seconds
Return code: 0

And here is the code for student_work_parallel.d:

/*
student_work_parallel.d
Author: Vasudev Ram
Copyright 2016 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: http://jugad2.blogspot.com
Product Store: https://gumroad.com/vasudevram
Twitter: https://mobile.twitter.com/vasudevram
*/

import std.stdio;
import core.thread;
import std.parallelism;

struct Student {
    int number;
    void doSlowOperation() {
        writefln("The work of student %s has begun", number);
        // Wait for a while to simulate a long-lasting operation
        Thread.sleep(1.seconds);
        writefln("The work of student %s has ended", number);
    }
}

void main() {
    auto students =
        [ Student(1), Student(2), Student(3), Student(4) ];
    foreach (student; parallel(students)) {
        student.doSlowOperation();
    }
}

Here is the output from running it, under the control of the same command timing program:

$ python c:\util\time_command.py student_work_parallel
The work of student 1 has begun
The work of student 2 has begun
The work of student 3 has begun
The work of student 4 has begun
The work of student 1 has ended
The work of student 2 has ended
The work of student 3 has ended
The work of student 4 has ended
Command: student_work_parallel
Time taken: 1.09 seconds
Return code: 0

We can see that the parallel version runs 3 times faster than the sequential one. And though there was some fluctuation in the exact ratio, it was in this range (3:1) over the half-dozen tests that I ran.

Not bad, considering that the only differences between the two programs is that the parallel one imports std.parallelism and uses this line:

foreach (student; parallel(students)) {

instead of this one used by the sequential program:

foreach (student; students) {

Also, messages about tasks started later in the sequence, appear before earlier tasks have ended, which shows that things are running in parallel, in the second program.

So (as the docs say) the parallel function from std.parallelism is a convenient high-level wrapper for some of the lower-level functionality of the parallelism module, and may suffice for some parallel processing use cases, without needing to do anything more. That is useful.

[1] The D standard library is called Phobos because Phobos is a moon of Mars (and hence a satellite of it. And Mars was the original name of the D language, given by its creator, Walter Bright (his company is/was called Digital Mars). But he says that many of his friends started calling it D (as it was like a better C), so he did too, and changed the name to D. So, Phobos (the library) is a "satellite" of D, a.k.a. Mars (the language) :) You can read an interview of Walter Bright (on the D blog) via this post:

Interview: Ruminations on D: Walter Bright, DLang creator

The post also has an HN thread about the interview. It also links to a video about D and other systems programming languages.

The Wikipedia article on Phobos is interesting. It says that the 'orbital motion of Phobos has been intensively studied, making it "the best studied natural satellite in the Solar System" in terms of orbits completed'. It also says where the name Phobos came from (a Greek god), and the source of the names of geographical features on Phobos (the book Gulliver's Travels). Also, Phobos has very low gravity (not even enough to keep it round in shape), so low that:

"A 68 kg (150 lb) person standing on the surface of Phobos would weigh the equivalent to about 60 g (2 oz) on Earth.[29]"

Maybe that is why the astronomers named some of the places on Phobos after Gulliver's Travels - because of Lilliput :)

The image at the top of the post is of Phobos and Deimos orbiting Mars.

- Vasudev Ram - Online Python training and consulting

Get updates on my software products / ebooks / courses.

Jump to posts: Python   DLang   xtopdf

Subscribe to my blog by email

My ActiveState recipes

FlyWheel - Managed WordPress Hosting

Share |

Video: C++, Rust, D and Go: Panel at LangNext '14

By Vasudev Ram

I recently came across this video of a panel discussion (in 2014) on modern systems programming languages. Yes, I know the term is controversial. Andrei and Rob say something about it. See the video.

The languages discussed are: C++, D, Go and Rust, by a panel at LangNext '14.

The panel consisted of key team members from the teams working on those languages:

- Bjarne Stroustrup for C++
- Andrei Alexandrescu for D
- Rob Pike for Go
- Niko Matsakis for Rust

(that's by language name in alphabetical order :)

Here is the video embedded below, and a link to it in case the embed does not work for you (sometimes happens).



I have only watched part of it so far (it is somewhat long), but found it interesting.

You can also download it with youtube-dl (written in Python, BTW) or some other video downloading tool, for watching offline. It's about 1.34 GB in size, so plan accordingly.

- Enjoy.

- Vasudev Ram - Online Python training and consulting
Follow me on Gumroad for product updates:

My Python posts     Subscribe to my blog by email

My ActiveState recipes

Share |