说明
本文是个人学习用的非官方中英对照译文,原文:https://diveintosystems.org/book/C14-SharedMemory/openmp.html。
原书采用 CC BY-NC-ND 4.0 许可(禁止演绎),请勿公开传播本译文。
标为「译注」「现代更新」的内容是译者补充,不代表原书作者观点;代码与原文逐字一致,未作改动。
14.7. Implicit Threading with OpenMP
到目前为止,我们介绍的都是用 POSIX 线程进行共享内存编程。尽管 Pthreads 很适合简单的应用程序,但随着程序本身越来越复杂,它也变得越来越难用。POSIX 线程是线程显式并行编程(explicit parallel programming)的一个例子,它要求程序员准确指定每个线程需要做什么,以及每个线程应当何时开始、何时停止。
Thus far, we have presented shared memory programming using POSIX threads. Although Pthreads are great for simple applications, they become increasingly difficult to use as programs themselves become more complex. POSIX threads are an example of explicit parallel programming of threads, requiring a programmer to specify exactly what each thread is required to do and when each thread should start and stop.
使用 Pthreads 时,要给现有的顺序程序逐步地加入并行也可能很困难。也就是说,程序员常常必须完全重写程序才能使用线程,而在试图并行化一个大型的现有代码库时,这往往是不可取的。
With Pthreads, it can also be challenging to incrementally add parallelism to an existing sequential program. That is, one must often rewrite the program entirely to use threads, which is often not desirable when attempting to parallelize a large, existing codebase.
开放多处理(Open Multiprocessing,OpenMP)库实现了 Pthreads 的一种隐式替代方案。OpenMP 内置于 GCC 以及 LLVM 和 Clang 等其他流行的编译器中,可以与 C、C++ 和 Fortran 编程语言一起使用。OpenMP 的一个主要优点是,它让程序员只需在代码的某些部分加入编译指示(pragma,即特殊的编译器指令),就能并行化现有的顺序 C 代码中的组成部分。OpenMP 专用的编译指示以 #pragma omp 开头。
The Open Multiprocessing (OpenMP) library implements an implicit alternative to Pthreads. OpenMP is built into GCC and other popular compilers such as LLVM and Clang, and can be used with the C, C++, and Fortran programming languages. A key advantage of OpenMP is that it enables programmers to parallelize components of existing, sequential C code by adding pragmas (special compiler directives) to parts of the code. Pragmas specific to OpenMP begin with
#pragma omp.
译注 · Mac 上的 OpenMP
macOS 自带的 Apple Clang 不直接支持 OpenMP:译者在 Mac 上用 clang -fopenmp 编译一个带 #pragma omp parallel 的程序,得到「unsupported option ‘-fopenmp’」错误。要在 Mac 上跟做本节,需要另外安装 OpenMP 运行时库(例如通过 Homebrew 安装 libomp)并相应调整编译选项,或者安装 GCC / 上游 LLVM,也可以直接在 Linux 中运行。
对 OpenMP 的详细介绍超出了本书的范围,不过我们会介绍一些常用的编译指示,并展示如何在一些示例应用程序中使用其中的几个。
Detailed coverage of OpenMP is outside the scope of this book, but we cover some common pragmas, and show how several can be used in the context of some sample applications.
14.7.1. 常用的编译指示
14.7.1. Common Pragmas
下面是 OpenMP 程序中最常用的一些编译指示:
Here are some of the most commonly used pragmas in OpenMP programs:
#pragma omp parallel这个编译指示创建一组线程,让每个线程运行其作用域中的代码(通常是一个函数调用)。调用这个编译指示通常相当于调用我们最初讨论 Pthreads 时介绍的
pthread_create和pthread_join函数对。这个编译指示可以带若干子句,包括:num_threads指定要创建的线程数。private是一个变量列表,列出对每个线程来说应当是私有(或局部)的变量。对线程私有的变量也可以在编译指示的作用域内声明(例子见下文)。每个线程都会得到每个这类变量的一份副本。shared是一个变量列表,列出应当在线程之间共享的变量。这类变量只有一份副本,由所有线程共享。default表示是否由编译器决定哪些变量应当共享。在大多数情况下,我们希望使用default(none),并显式指定哪些变量应当共享、哪些应当私有。
#pragma omp parallelThis pragma creates a team of threads and has each thread run the code in its scope (usually a function call) on each thread. An invocation of this pragma is usually equivalent to an invocation of the
pthread_createandpthread_joinfunction pairing discussed in our original discussion on Pthreads. The pragma may have a number of clauses, including the following:
num_threadsspecifies the number of threads to create.
privateis a list of variables that should be private (or local) to each thread. Variables that should be private to a thread can also be declared within the scope of the pragma (see below for an example). Each thread gets its own copy of each variable.
sharedis a listing of variables that should be shared amongst the threads. There is one copy of the variable that is shared amongst all threads.
defaultindicates whether the determination of which variables should be shared is left up to the compiler. In most cases, we want to usedefault(none)and specify explicitly which variables should be shared, and which should be private.
#pragma omp for指定每个线程执行
for循环迭代的一个子集。尽管循环的调度由系统决定,但默认通常是在标量乘法的例子中首次讨论的「分块」方法。这是一种静态的调度形式:每个线程得到一个分配好的块,然后处理块中的迭代。不过,OpenMP 也让动态调度变得很容易。在动态调度中,每个线程得到若干次迭代,处理完之后再请求一组新的迭代。调度策略可以用下面的子句来设置:schedule(dynamic):指定应当使用动态形式的调度。尽管在某些情况下这样做有好处,但静态(默认)形式的调度通常更快。
#pragma omp forSpecifies that each thread execute a subset of iterations of a
forloop. Although the scheduling of the loops is up to the system, the default is usually the “chunking” method first discussed in the scalar multiplication example. This is a static form of scheduling: each thread gets an assigned chunk, and then processes the iterations in its chunk. However, OpenMP also makes dynamic scheduling easy. In dynamic scheduling, each thread gets a number of iterations, and requests a new set upon completing processing their iteration. The scheduling policy can be set using the following clause:
schedule(dynamic): specifies that a dynamic form of scheduling should be used. While this is advantageous in some cases, the static (default) form of scheduling is usually faster.
#pragma omp parallel for这个编译指示是
omp parallel和omp for编译指示的组合。与omp for编译指示不同,omp parallel for编译指示还会先生成一组线程,然后再给每个线程分配循环迭代的一个集合。
#pragma omp parallel forThis pragma is a combination of the
omp paralleland theomp forpragmas. Unlike theomp forpragma, theomp parallel forpragma also generates a team of threads before assigning each thread a set of iterations of the loop.
#pragma omp critical这个编译指示用来指定其作用域下的代码应当被视为临界区,即一次只能有一个线程执行这段代码,以确保正确的行为。
#pragma omp criticalThis pragma is used to specify that the code under its scope should be treated as a critical section — that is, only one thread should execute the section of code at a time to ensure correct behavior.
还有几个线程可以调用的函数,在执行中常常很有用。例如:
There are also several functions that a thread can access that are often useful for execution. For example:
omp_get_num_threads返回当前正在执行的线程组中的线程数。
omp_get_num_threadsreturns the number of threads in the current team that is being executed.
omp_set_num_threads设置一个线程组应当拥有的线程数。
omp_set_num_threadssets the number of threads that a team should have.
omp_get_thread_num返回调用线程的标识符。
omp_get_thread_numreturns the identifier of the calling thread.
原书警示 · omp parallel for 指令只适用于 for 循环!(The omp parallel for directive works only with for loops!)
记住,omp parallel for 编译指示只适用于 for 循环。其他类型的循环,例如 while 循环和 do-while 循环,都不受支持。
Keep in mind that the
omp parallel forpragma works only withforloops. Other types of loops, such aswhileloops anddo-whileloops, are not supported.
14.7.2. 你好,线程:OpenMP 风味
14.7.2. Hello Threading: OpenMP flavored
我们再来看「Hello World」(hellothreads.c)程序,这次用 OpenMP 而不是 Pthreads 来写:
Let’s revisit our “Hello World” (hellothreads.c) program, now using OpenMP instead of Pthreads:
1 | #include <stdio.h> |
注意,OpenMP 程序比 Pthreads 版本短得多。为了使用 OpenMP 库函数,我们包含头文件 omp.h。main 中的 omp parallel num_threads(nthreads) 编译指示创建一组线程,每个线程都调用 HelloWorld 函数。子句 num_threads(nthreads) 指定总共应当生成 nthreads 个线程。这个编译指示还会把每个创建的线程合并回单线程进程。换句话说,创建和合并线程的所有底层工作都对程序员抽象掉了,只需加入一个编译指示就能完成。因此,OpenMP 被认为是一种隐式多线程(implicit threading)库。
Note that the OpenMP program is much shorter than the Pthreads version. To access the OpenMP library functions, we include the header file
omp.h. Theomp parallel num_threads(nthreads)pragma inmaincreates a set of threads, where each thread calls theHelloWorldfunction. The clausenum_threads(nthreads)specifies that a total ofnthreadsshould be generated. The pragma also joins each created thread back to a single-threaded process. In other words, all the low-level work of creating and joining threads is abstracted away from the programmer and is accomplished with the inclusion of just one pragma. For this reason, OpenMP is considered an implicit threading library.
OpenMP 还省去了显式管理线程 ID 的需要。在 HelloWorld 中,omp_get_thread_num 函数取出运行它的那个线程的唯一 ID。
OpenMP also abstracts away the need to explicitly manage thread IDs. In the context of
HelloWorld, theomp_get_thread_numfunction extracts the unique ID associated with the thread that is running it.
编译代码
Compiling the code
我们给编译器传入 -fopenmp 选项来编译并运行这个程序,这个选项表示我们是在用 OpenMP 进行编译:
Let’s compile and run this program by passing the
-fopenmpflag to the compiler, which signals that we’re compiling with OpenMP:
1 | $ gcc -o hello_mp hello_mp.c -fopenmp |
由于线程的执行在后续运行中可能变化,重新运行这个程序会得到不同顺序的消息:
Since the execution of threads can change with subsequent runs, rerunning this program results in a different sequence of messages:
1 | $ ./hello_mp 4 |
这种行为与我们用 Pthreads 编写的例子是一致的。
This behavior is consistent with our example with Pthreads.
14.7.3. 一个更复杂的例子:OpenMP 版计数排序
14.7.3. A More Complex Example: CountSort in OpenMP
OpenMP 的一个强大优势在于,它让程序员能够逐步地并行化自己的代码。为了实际看一看,我们来并行化本章前面讨论过的、更复杂的计数排序算法(串行代码在这里:countSort.c)。回想一下,这个算法对只包含少量取值的数组进行排序。串行程序的 main 函数如下:
A powerful advantage of OpenMP is that it enables programmers to incrementally parallelize their code. To see this in action, let’s parallelize the more complex CountSort algorithm discussed earlier in this chapter (the serial code is located here: countSort.c). Recall that this algorithm sorts arrays containing a small range of values. The main function of the serial program looks like the following:
1 | int main( int argc, char **argv ) { |
main 函数在做了一些命令行解析并生成一个随机数组之后,先调用 countsElems 函数,再调用 writeArray 函数。
The
mainfunction, after doing some command line parsing and generating a random array, calls thecountsElemsfunction followed by thewriteArrayfunction.
用 OpenMP 并行化 CountElems
Parallelizing CountElems Using OpenMP
并行化前面的程序有好几种方法。一种方法(如下面的例子所示)是在 countElems 和 writeArray 函数中使用 omp parallel 编译指示。这样,就不需要对 main 函数做任何改动。程序的完整版本见:countSort_mp.c。
There are several ways to parallelize the preceding program. One way (shown in the example that follows) uses the
omp parallelpragma in the context of thecountElemsandwriteArrayfunctions. As a result, no changes need to be made to themainfunction. A full version of the program is available at: countSort_mp.c.
首先,我们来看如何用 OpenMP 并行化 countElems 函数:
First, let’s examine how to parallelize the
countElemsfunction using OpenMP:
1 | void countElems(int *counts, int *array, long length) { |
在这个版本的代码中,使用了三个编译指示:
In this version of the code, three pragmas are employed:
#pragma omp parallel编译指示表示应当创建一组线程。main中的omp_set_num_threads(nthreads)这一行把线程组的默认大小设为nthreads。如果没有使用omp_set_num_threads函数,分配的线程数将等于系统中的核心数。提醒一下,omp parallel编译指示在块的开头隐式地创建线程,在块的末尾合并它们。花括号({})用来指定作用域。shared子句声明变量counts、array和length在所有线程之间共享(全局的)。因此,变量val、i和local[MAX]在每个线程中都是局部声明的。
- The
#pragma omp parallelpragma indicates that a team of threads should be created. Theomp_set_num_threads(nthreads)line inmainsets the default size of the thread team to benthreads. If theomp_set_num_threadsfunction is not used, then the number of threads assigned will equal the number of cores in the system. As a reminder, theomp parallelpragma implicitly creates threads at the beginning of the block and joins them at the end of the block. Braces ({}) are used to specify scope. Thesharedclause declares that the variablescounts,array, andlengthare shared (global) among all the threads. Thus, the variablesval,i, andlocal[MAX]are declared locally in each thread.
- 下一个编译指示是
#pragma omp for,它并行化for循环,把迭代次数分给各个线程。OpenMP 会计算如何最好地划分循环的迭代。如前所述,默认策略通常是分块方法,让每个线程计算大致相同数量的迭代。因此,每个线程读取共享数组array的一部分,并在它的局部数组local中累计计数。
- The next pragma is
#pragma omp for, which parallelizes theforloop, splitting the number of iterations among the number of threads. OpenMP calculates how best to split up the iterations of the loop. As previously mentioned, the default strategy is usually a chunking method, wherein each thread gets roughly the same number of iterations to compute. Thus, each thread reads a component of the shared arrayarray, and accumulates its counts in its local arraylocal.
#pragma omp critical编译指示表示临界区作用域中的代码一次只能由一个线程执行。这相当于这个程序的 Pthreads 版本中使用的互斥锁。在这里,每个线程依次递增共享的counts数组。
- The
#pragma omp criticalpragma indicates that the code in the scope of the critical section should be executed by exactly one thread at a time. This is equivalent to the mutex that was employed in the Pthreads version of this program. Here, each thread increments the sharedcountsarray one at a time.
我们用 1 亿个元素运行这个函数,来感受一下它的性能:
Let’s get a sense of the performance of this function by running it with 100 million elements:
1 | $ ./countElems_mp 100000000 1 |
这是非常出色的性能:我们的函数在两个线程上得到了 2 的加速比,在四个线程上得到了 3.63 的加速比。性能甚至比 Pthreads 的实现还要好!
This is excellent performance, with our function getting a speedup of 2 on two threads, and a speedup of 3.63 on four threads. We get even better performance than the Pthreads implementation!
OpenMP 中的 writeArray 函数
The writeArray Function in OpenMP
并行化 writeArray 函数要难得多。下面的代码展示了一种可能的解决方案:
Parallelizing the
writeArrayfunction is much harder. The following code shows one possible solution:
1 | void writeArray(int *counts, int *array) { |
在并行化之前,我们对这个函数做了一处修改,因为旧版本的 writeArray 使 j 依赖于循环的前几次迭代。在这个版本中,每个线程根据 counts 中前面所有元素之和,算出自己唯一的 start 值。
Prior to parallelizing, we made a change to this function, because the old version of
writeArraycausedjto have a dependency on the previous iterations of the loop. In this version, each thread calculates its uniquestartvalue based on the sum of all the previous elements incounts.
去掉这个依赖之后,并行化就相当直接了。#pragma omp parallel for 编译指示生成一组线程,并通过给每个线程分配循环迭代的一个子集来并行化 for 循环。提醒一下,这个编译指示是 omp parallel 和 omp for 编译指示(在并行化 countElems 时用过)的组合。
When this dependency is removed, the parallelization is pretty straightforward. The
#pragma omp parallel forpragma generates a team of threads and parallelizes theforloop by assigning each thread a subset of the iterations of the loop. As a reminder, this pragma is a combination of theomp paralleland theomp forpragmas (which were used in the parallelization ofcountElems).
分块的线程调度方法(如前面的 countElems 函数中所示)在这里并不合适,因为 counts 中各个元素的频次可能相差悬殊。因此,各线程的工作量不会相等,导致一些线程分到的工作比其他线程多。所以这里使用了 schedule(dynamic) 子句,让每个线程完成分配给它的迭代后,再向线程管理器请求新的迭代。
A chunking approach to scheduling threads (as shown in the earlier
countElemsfunction) is not appropriate here, because it is possible that each element incountshas a radically different frequency. Therefore, the threads will not have equal work, resulting in some threads being assigned more work than others. Therefore, theschedule(dynamic)clause is employed, so that each thread completes the iteration it is assigned before requesting a new iteration from the thread manager.
由于每个线程写的是数组中不同的位置,这个函数不需要互斥。
Since each thread is writing to distinct array locations, mutual exclusion is not needed for this function.
注意,OpenMP 代码比 POSIX 线程的实现简洁得多。代码非常易读,而且只需要很少的修改。这就是抽象(abstraction)的威力之一:实现细节对程序员隐藏了起来。
Notice how much cleaner the OpenMP code is than the POSIX thread implementation. The code is very readable, and required very little modification. This is one of the powers of abstraction, in which the implementation details are hidden from the programmer.
然而,抽象必然要以控制力为代价。程序员假定编译器足够「聪明」,能处理并行化的种种细节,因而能更轻松地并行化自己的应用程序。但是,程序员不再对并行化的细节做出具体的决定。如果不清楚 OpenMP 编译指示在底层是如何执行的,就可能很难调试 OpenMP 应用程序,也很难知道在某个时候使用哪个编译指示最合适。
However, a necessary trade-off for abstraction is control. The programmer assumes that the compiler is “smart” enough to take care of the particulars of parallelization and thus has an easier time parallelizing their application. However, the programmer no longer makes detailed decisions about the particulars of that parallelization. Without a clear idea of how OpenMP pragmas execute under the hood, it can be difficult to debug an OpenMP application or know which pragma is the most appropriate to use at a given time.
14.7.4. 进一步学习 OpenMP
14.7.4. Learning More about OpenMP
对 OpenMP 更深入的讨论超出了本书的范围,不过有一些学习 OpenMP 的免费资源1,2很有用。
A deeper discussion of OpenMP is beyond the scope of this book, but there are useful free resources1,2 for learning OpenMP.
现代更新(截至 2026-11)· OpenMP 6.0
2024 年 11 月 14 日,OpenMP 架构审查委员会(ARB)在 SC24 大会上发布了 OpenMP 6.0 规范。按官方新闻稿,主要新增内容包括:简化任务(task)编程,例如允许不属于任何线程组的「自由代理线程」(free-agent thread)执行任务、可以录制任务图以便高效重放;增强加速器设备支持;完整支持 C23、C++23 和 Fortran 2023 等最新语言标准;以及让用户更细致地控制存储资源和内存空间。本节介绍的 parallel、for、critical 等基本编译指示在新版本中依然适用。Business Wire:OpenMP ARB Releases OpenMP 6.0 for Easier Programming
参考文献
References:
Blaise Barney. “OpenMP”. https://hpc.llnl.gov/tuts/openMP/
Richard Brown and Libby Shoop. “Multicore Programming with OpenMP”. CSinParallel: Parallel Computing in the Computer Science curriculum. http://selkie.macalester.edu/csinparallel/modules/MulticoreProgramming/build/html/index.html