Using Socat to Simulate Networking Traffic to Test and Debug

If you don’t know socat, you probably should. From its man:

Socat is a command line based utility that establishes two bidirectional byte streams and transfers data between them. Because the streams can be constructed from a large set of different types of data sinks and sources (see address types), and because lots of address options may be applied to the streams, socat can be used for many different purposes.

Socat can do a lot of things. I will now write about a problem I had, and show how I used socat to make my life easier.

The Problem

Financial institutions like credit card brands, issuer banks, acquirers, atm networks, etc across the globe usually do all their online communication over tcp/ip using some flavor of the iso8583 “standard” protocol. The quotes indicate how much custom stuff these institutions add to the standard. I used to code lots of interfaces to enable interaction between such kinds of institutions, and deviation from said iso8583 standard would usually come in a document file agreed upon both ends to enable communication; and would specify changes such as, add extra types of messages (credit card purchase, cancelation, etc), extra fields (like the address of the cardholder), extra types of messages (credit card purchase, cancelation, etc), a different encoding for some of the fields (ASCII, EBCDIC, BCD, binary, etc), use different codes for responses (acepted, rejected, stolen card), different length for a field, different padding, etc. 99% of the time, there is no access to any kind of simulator for “the other end”, so performing the integration testing can be a nightmare. The team on the other end usually speaks another language, lives in a different timezone, etc. The try/debug/feedback/repeat loop takes forever. And writting your own simulator was out of the question, as it would take a lot of time to write a new simulator for each integration, and writing a generic simulator to be able to be configured for any possible protocol would be a big project by itself. We need a way to test/debug that the interface we programmed to enable our system communicate with the other end, can satisfy all the use cases. For brevity’s sake, we will have only one such case, and invent a very simple protocol as a proof of concept.

The Solution

What we need is an ad-hoc pseudo-chat client that lets us send arbitrary payloads over a persistent tcp connection, while being as verbose as possible. Uhmmm… What? A simulator! We want a program that:

• Can connect to our system via tcp socket.
• Can persist that connection. (the protocol can have “handshaking” logic for each connection)
• Lets us send arbitrary payloads
• Accept arbitrary payloads and print them.
• Print those also in hexa in case the payload is not printable.

Why? Well… if we have everything above done for us, human insteraction can do the rest! How? Wait for it… With socat of course! Here is an example: Let’s say our system is an ATM, and we are going to be able to send transactions to a new banking institution. For that, we have to implement their Bacon&Eggs protocol. They don’t provide a simulator for their end, but they at least give us some example payloads. Here is a bacon payload (request) with id 666:

__ _.._ .-'__`-._.'.--.'.__., /--' '-._.' '-._./ id=666 /__.--._.--._.'``-.__/ '._.-'-._.-._.-''-..' 

And here is an eggs payload (response, referencing the request with id 666):

,-""""-. ,'"`. / `. / \ | ); : responded=666 \ ,' : ; `-....-' `.___,' 

  Ok, we upgrade our ATM software to support protocol Bacon&Eggs. How can we test and debug as we go?

Setup

1. Create a pipe to receive the payloads. mkfifo /tmp/receive
2. Create a pipe to send the response payloads. mkfifo /tmp/send
3. Create a file to stream the responses to the sending pipe (more on this later). touch /tmp/stream
4. Set a console to read (and keep reading) from /tmp/receive. tail -f /tmp/receive
5. Set a console to write (and keep writing) to /tmp/send. tail -f /tmp/stream >> /tmp/send
6. Set socat to listen and accept connections on some port and:
7. whatever it receives from the socket, it forwards it to the receiving pipe.
8. whatever it receives from the sending pipe, it forwards it to the socket.
9. print the payload in text and in hexa

Like this: socat -x -v -d -d tcp4-listen:8032,fork,reuseaddr pipe:/tmp/send\!\!/tmp/receive And thats it! Now we can make our ATM connect to port 8032 at localhost as if it were a true Bacon&Eggs system. Now, whenever our ATM software sends a bacon request with id=667, we just open our egg payload response example in a text or hex editor, replace 666 with 667 to make it a correct response and send it back. cat eggs.payload >> /tmp/stream

BOOM!

Our ATM software receives a correct egg response for its bacon request and thus the transaction is approved. The connection is still open, so we can keep sending requests and responses and try different scenarios and use cases. It’s like chatting with your software over a socket. Here is a screenshot of my human operated Bacon&Eggs protocol server processing a transaction.