[풀스택] Zero message queue (ZMQ)

 

 

 

ZeroMQ Patterns

• Request-reply, which connects a set of clients to a set of services.
  This is a remote procedure call and task distribution pattern (REQ, REP, DEALER, ROUTER).
  • 요청하면 응답을 보내준다. 
  • 내가 요청하면 가운데 애가 받아서,
    일꾼이 3명 있는데 어떤 애한테 보낼까?
    니가 가져가라 등등 적절히 분배하는 과정도 이미 구현됨
    이 요청에 대해서는 니가 처리했음 좋겠다 -> 설정하는 것도 구현됨
•  Pub-sub, which connects a set of publishers to a set of subscribers.
  This is a data distribution pattern (PUB, SUB, XPUB, XSUB).
  • 가입한 애들한테 정보를 뿌림
• Pipeline, which connects nodes in a fan-out/fan-in pattern that can have multiple steps and loops.
  This is a parallel task distribution and collection pattern (PULL, PUSH).
  • 대규모의 병렬처리를 동시다발적으로 하는 것 -> OS's 파이프라인
• Exclusive pair, which connects two sockets exclusively.
  This is a pattern for connecting two threads in a process, not to be confused with “normal” pairs of sockets.

 

 

 

ZeroMQ Sockets

- 기존의 사고 방식 - 1:1 연결을 만들 때, TCP or UDP -> 이외의 판단은 X
- 1:N, N:M 구현은 개발자의 몫
- 소켓이 해주는 것은 없다 

- ZMQ는 이미 다 구현되어 있음

 

• Conventional sockets -> 전통적인 방식의 소켓
  • Synchronous interface to either connection-oriented reliable byte streams (SOCK_STREAM), or connection- less unreliable datagrams (SOCK_DGRAM)
  • Transfer streams of bytes or discrete datagrams Allow 1:1 (two peers), N:1 (many clients, one server),
  • or in some cases 1:N (multicast) relationships
• ZeroMQ sockets -> 제로 메세지 큐의 소켓 
  • Abstraction of an asynchronous message queue, with the exact queueing semantics depending on the socket type in use
  • Transfer discrete messages
  • Allow many-to-many relationships

 

•  Asynchronous ZeroMQ sockets
  • Timings of the physical connection setup and tear down, reconnect and effective delivery are transparent to the user and organized by ZeroMQ
  • Further, messages may be queued in the event that a peer is unavailable to receive them
• 제로 메세지 큐
  어플리케이션 입장에서 커넥션이 밑에서 실제로 
  열려있든 말든, 메세지 주고 받을 수 있다.
  tcp 연결 설정 -> 끊어지면 정보를 주고 받을 수 없다
  
  연결 요청이 아직 안 되어도
  application 데이터를 보낼 수 있다
  언젠가 연결 요청이 이루어질 것이라는 전제 하에,
  메세지 큐에 쌓아놓고, 언젠가는 보낸다
  
  연결이 된 상태에서 이동하거나, 네트워크가 불안하여 연결이 끊어짐
  -> 어플리케이션이 영향을 받지 않고, 정보를 보낼 수 있다.
  zmq가 연결 재시도 함 + app이 보내는 정보를 계속 쌓아둠
  
  연결이 끊어졌는지 , 안끊어졌는지 상관 없이 사용 가능
  무선을 사용하는 환경에서 zmq 장점 !

 

•  ZeroMQ socket lifetime (similar to BSD sockets)
  • Creating and destroying sockets, which go together to form a karmic circle of socket life
  • Configuring sockets by setting options on them and checking them if necessary
  • Plugging sockets into the network topology by creating ZeroMQ connections to and from them.
  • Using the sockets to carry data by writing and receiving messages on them.

 

 

 

ZeroMQ Sockets bind & connect

It doesn’t matter who binds and who connects.
In general, the server used Bind while the client used Connect
ZeroMQ creates queues per underlying connection.
If your socket is connected to three peer sockets, then there are three messages queues behind the scenes.

With Bind, you allow peers to connect to you, (서버의 역할을 하는 것)
thus you don’t know how many peers there will be in the future and you cannot create the queues in advance.
Instead, queues are created as individual peers connect to the bound socket.

 

With Connect, in general cases, ZeroMQ knows that there’s going to be at least a single peer and thus it can create a single queue immediately.

 

• bind - server / connect - client
• When should I use bind:
  • As a general rule use bind from the most stable points in your architecture,
    and use connect from dynamic components with volatile endpoints
  • For request/reply, the service provider might be the point where you bind and the clients are using connect.
    Just like plain old TCP
  • 웹 서버는 24시간 살아있음 -> stable -> 이런 애를 바인드 시킨다. 
    웹 브라우저 -> (웹서버)한테 connect 하면 되겠네 
• When should I use connect:
  • If you can’t figure out which parts are more stable (i.e. peer-to-peer),
    consider a stable device in the middle, which all sides can connect to
  • 기존 tcp/ip에는 노드와 노드를 tcp 소켓 줄 하나로 연결
    얘는 어플리케이션 입장에서 다양한 줄을 가질 수 있고, 
    하나는 바인드, 하나는 커넥트 이런 식으로 써도 된다. 

 

 

 

RabbitMQ (open source message broker)

ZMQ 누구는 반드시 서버다 이런 룰이 없다
분산적으로 노드가 대등한 관계에서 정보를 주고 받는 구조 

RabbitMQ -> message broker 브로커가 있는 구조

 

중앙 집중형 Broker 기반 메세징 개념

• 정보가 큐에 차곡차곡 쌓이고, 얘가 메세지를 밀어냄
• 사람들이 몰리면 - 천천히
  내 요청만 있으면 - 바로 신속히
• subscribe로 보내기도 하고, 동시 다발적으로 (fan out) 쫙 뿌림
• ex) 날씨 위젯 - 피드백 없이 일반적인 전송을 할 때 or 알람을 보내거나 등등