Time to ditch JSON from our stack

Lately i'm doing work on crypto financial industry and creating a bunch of connectors to each exchange using websocket and http rest api. Finally i realize that we need to reconsider why we are using text encoder in the first place. Why we are not choosing more efficient encoding. Lots of crypto exchange range from coinbase, binance, ftx use JSON as their websocket stream. And it's consider the fastest and reliable way to get message across. That's scary.

We might not think that this one is a big deal. As the cost is not visible if we are not seeing that closer. If we profile our software most of the works will spent on receiving, serializing and deserializing message. How can we produce low latency software if our encoder is in JSON.

Let's consider the conventional financial world like US stock market, NASDAQ and others. They use FIX, well it's still text and human readable but it's better than JSON. And right now the whole industry is moving to SBE as their standard which is the optimal binary serializer for streaming data especially optimized for financial market and exchange. For sure if you are doing market making you will need more than TCP/IP but that's better than the state of crypto exchange right now.

The crypto exchange industry is still young in term of technology and also needs more technologist to see what can be improved even that's not the real concern for now. But as educated software engineer we can create awareness about this. How we can tolerate this waste generated in our industry.

I hope this bring some light into our industry and our responsibility as engineer. Software engineer is noble profession, we might not have our own oath like other profession have, but we owe that to our self and our generation to not generate waste as it's very easy to type inefficient code just in the matters of seconds.