MPI中的缓冲区和非阻塞通信

转载自: Introduction to MPI - Part II (Youtube)

Buffering 

Suppose we have

if(rank==0)
    MPI_Send(sendbuf,...,1,...)
if(rank==1)
    MPI_Recv(recvbuf,...,0,...)

These are blocking communications, which means they will not return until the arguments to the functions can be safely modified by subsequent statements in the program.

Assume that process 1 is not ready to receive. Then there are 3 possibilities for process 0:

(1) stops and waits until process 1 is ready to receive

(2) copies the message at sendbuf into a system buffer (can be on process 0, process 1 or somewhere else) and returns from MPI_Send

(3) fails

As long as buffer space is available, (2) is a reasonable alternative.

An MPI implementation is permitted to copy the message to be sent into internal storage, but it is not required to do so.

What if not enough space is available?

>> In applications communicating large amounts of data, there may not be enough momory (left) in buffers.

>> Until receive starts, no place to store the send message.

>> Practically, (1) results in a serial execution.

A programmer should not assume that the system provides adequate buffering.

Now consider a program executing:

Process 0 Process 1
MPI_Send to process 1 MPI_Send to process 0
MPI_Recv from process 1 MPI_Recv from process 0

Such a program may work in many cases, but it is certain to fail for message of some size that is large enough.

There are some possible solutions:

>> Ordered send and receive - make sure each receive is matched with send in execution order across processes.

>> The aboved matched pairing can be difficult in complex applications. An alternative is to use MPI_Sendrecv. It performs both send and receive such that if no buffering is available, no deadlock will occur.

>> Buffered sends. MPI allows the programmer to provide a buffer into which data can be placed until it is delivered (or at lease left in buffer) via MPI_Bsend.

>> Nonblocking communication. Initiated, then program proceeds while the communication is ongoing, until a check that communication is completed later in the program. IMPORTANT: must make certain data not modified until communication has completed.

Safe programs

>> A program is safe if it will produce correct results even if the system provides no buffering.

>> Need safe programs for portability.

>> Most programmers expect the system to provide some buffering, hence many unsafe MPI programs are around.

>> Write safe programs using matching send with receive, MPI_Sendrecv, allocating own buffers, nonblocking operations.

Nonblocking communications

>> nonblocking communications are useful for overlapping communication with computation, and ensuring safe programs.

>> a nonblocking operation request the MPI library to perform an operation (when it can).

>> nonblocking operations do not wait for any communication events to complete.

>> nonblocking send and receive: return almost immediately

>> can safely modify a send (receive) buffer only after send (receive) is completed.

>> "wait" routines will let program know when a nonblocking operation is done.

Example - Communication between processes in ring topology

>> With blocking communications it is not possible to write a simple code to accomplish this data exchange.

>> For example, if we have MPI_Send first in all processes, program will get stuck as there will be no matching MPI_Recv to send data.

>> Nonblocking communication avoids this problem.

 1 #include <stdio.h>
 2 #include <stdlib.h>
 3 #include "mpi.h"
 4
 5 int main(int argc, char *argv[]) {
 6     int numtasks, rank, next, prev, buf[2], tag1=1, tag2=2;
 7
 8     tag1=tag2=0;
 9     MPI_Request reqs[4];
10     MPI_Status stats[4];
11
12     MPI_Init(&argc, &argv);
13     MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
14     MPI_Comm_rank(MPI_COMM_WORLD, &rank);
15
16     prev= rank-1;
17     next= rank+1;
18     if(rank == 0)    prev= numtasks - 1;
19     if(rank == numtasks-1) next= 0;
20     MPI_Irecv(&buf[0], 1, MPI_INT, prev, tag1, MPI_COMM_WORLD, &reqs[0]);
21     MPI_Irecv(&buf[1], 1, MPI_INT, next, tag2, MPI_COMM_WORLD, &reqs[1]);
22     MPI_Isend(&rank, 1, MPI_INT, prev, tag2, MPI_COMM_WORLD, &reqs[2]);
23     MPI_Isend(&rank, 1, MPI_INT, next, tag1, MPI_COMM_WORLD, &reqs[3]);
24     MPI_Waitall(4, reqs, stats);
25
26     printf("Task %d communicated with tasks %d & %d\n",rank,prev,next);
27     MPI_Finalize();
28     return 0;
29 }

Summary for Nonblocking Communications

>> nonblocking send can be posted whether a matching receive has been posted or not.

>> send is completed when data has been copied out of send buffer.

>> nonblocking send can be matched with blocking receive and vice versa.

>> communications are initiated by sender

>> a communication will generally have lower overhead if a receive buffer is already posted when a sender initiates a communication.

时间: 2024-11-06 16:00:38

MPI中的缓冲区和非阻塞通信的相关文章

用Java实现非阻塞通信

用ServerSocket和Socket来编写服务器程序和客户程序,是Java网络编程的最基本的方式.这些服务器程序或客户程序在运行过程中常常会阻塞.例如当一个线程执行ServerSocket的accept()方法时,假如没有客户连接,该线程就会一直等到有了客户连接才从accept()方法返回.再例如当线程执行Socket的read()方法时,如果输入流中没有数据,该线程就会一直等到读入了足够的数据才从read()方法返回. 假如服务器程序需要同时与多个客户通信,就必须分配多个工作线程,让它们分

Java_nio_非阻塞通信

1.常用支持非阻塞通信的类 ServerSocketChannel: ServerSocket 的替代类, 支持阻塞通信与非阻塞通信. SocketChannel: Socket 的替代类, 支持阻塞通信与非阻塞通信. Selector: 为ServerSocketChannel 监控接收连接就绪事件, 为 SocketChannel 监控连接就绪, 读就绪和写就绪事件. SelectionKey: 代表 ServerSocketChannel 及 SocketChannel 向 Selecto

NIO实现TCP的非阻塞通信

这一次写NIO实现非阻塞通信时遇到了很多问题,我所理解的非阻塞是对于一个用户而言它的读写不会相互制约,而在此次编写过程中,发现其实非阻塞是相对于多个用户而言的. 看到网上一个对同步异步阻塞非阻塞的例子,感觉挺好的,就拷过来了: 老张爱喝茶,废话不说,煮开水. 出场人物:老张,水壶两把(普通水壶,简称水壶:会响的水壶,简称响水壶). 1 老张把水壶放到火上,立等水开.(同步阻塞) 老张觉得自己有点傻 2 老张把水壶放到火上,去客厅看电视,时不时去厨房看看水开没有.(同步非阻塞) 老张还是觉得自己有

【python】网络编程-SocketServer 实现客户端与服务器间非阻塞通信

利用SocketServer模块来实现网络客户端与服务器并发连接非阻塞通信.首先,先了解下SocketServer模块中可供使用的类:BaseServer:包含服务器的核心功能与混合(mix-in)类挂钩:这个类只用于派生,所以不会生成这个类的实例:可以考虑使用TCPServer和UDPServer.TCPServer/UDPServer:基本的网络同步TCP/UDP服务器.UnixStreamServer/ UnixDatagramServer:基本的基于文件同步TCP/UDP服务器.Fork

基于MFC的socket编程(异步非阻塞通信)

对于许多初学者来说,网络通信程序的开发,普遍的一个现象就是觉得难以入手.许多概念,诸如:同步(Sync)/异步(Async),阻塞(Block)/非阻塞(Unblock)等,初学者往往迷惑不清,只知其所以而不知起所以然. 异步方式指的是发送方不等接收方响应,便接着发下个数据包的通信方式:而同步指发送方发出数据后,等收到接收方发回的响应,才发下一个数据包的通信方式. 阻塞套接字是指执行此套接字的网络调用时,直到成功才返回,否则一直阻塞在此网络调用上,比如调用recv()函数读取网络缓冲区中的数据,

非阻塞算法在并发容器中的实现【转】

转自:https://www.ibm.com/developerworks/cn/java/j-lo-concurrent/ 非阻塞算法在 Java 中的应用越来越广泛 , ConcurrentLinkedQueue 是 java. concurrent 包中基于非阻塞算法实现的并发容器的典范.通过本文,您将了解非阻塞算法的工作原理及其在 ConcurrentLinkedQueue 中的具体实现机制. 简介 非阻塞算法在更细粒度的层面协调争用,它比传统的锁有更高的并发性.随着非阻塞算法在 Jav

同步与异步,阻塞与非阻塞基础

1.同步与异步 同步和异步关注的是消息通信机制 (synchronous communication/ asynchronous communication) 同步,在发出一个调用时,在没有得到结果之前,该调用就不返回.但是一旦调用返回,就得到返回值了.是由调用者主动等待这个调用的结果. 异步,调用在发出之后,这个调用就直接返回了,所以没有返回结果.当一个异步过程调用发出后,调用者不会立刻得到结果.而是在调用发出后,被调用者通过状态.通知来通知调用者,或通过回调函数处理这个调用. 2.阻塞与非阻

Java千百问_02基本使用(012)_如何编写非阻塞SocketChannel程序

点击进入_更多_Java千百问 1.如何编写非阻塞SocketChannel程序 了解Socket看这里:Socket是什么 了解 SocketChannel看这里:Socket.SocketChannel有什么区别 使用SocketChannel的最大好处就是可以进行非阻塞IO,每次链接后都会直接返回,不会阻塞线程.将需要多个线程的任务通过几个线程就能完成,降低了了性能消耗. 了解阻塞.非阻塞看这里:阻塞.非阻塞有什么区别 要编写SocketChannel,需要了解java.nio包中如下几个

客户端非阻塞socket建链流程

TCP协议是面向连接的.可靠的.基于字节流的传输层协议.那使用tcp协议进行通信的两端是如何进行通信的?使用tcp协议进行通信的两端是通过套接字(scoket)来建立连接的.套接字socket主要有两种类型,阻塞和非阻塞.通常为了防止进程阻塞以及避免cpu被长时间占用,客户端和服务端一般都会采用非阻塞socket进行通信,其中Nginx就是一个典型的例子.下面我们就以Nginx的upstream机制所涉及的与后端服务器建链的流程来总结下使用非阻塞socket的客户端建链流程. 先来看下Nginx