The REPL can save hours of pain through pairing with non-technical colleagues.

The boundary between a business user and a developer can often be a big source of waste. Going back and forth between the user and your code to discover what data is really needed can really cause pain and waste a huge amount of time. The REPL can be used as an interactive environment to pair with a user and write the exact code required, first time.

The Problem

Recently I was developing a middleware system where data would come in via HTTP as JSON, be converted into SOAP XML and sent to a finance system. As usual, the field names in the finance system were completely different from the system they being provided by. There were spreadsheets which attempted to define this mapping but the system was evolving and the spreadsheet was not always correct. For example, there were 3 different date fields that all sounded very similar, which one should be sent to the finance system?


There was a colleague (lets call him ‘M’) who understood how to use both systems and really knew what the data meant, he had the knowledge but he had no idea how to program. After a number of attempts at getting the mapping correct myself, each requiring a build, deploy and test cycle, I was getting a bit frustrated. I had a big screen with the Clojure mapping on one side and a REPL running on the other and I realized that anyone should be able to understand what was going on if they could see the input, the mapping and the output.

For anyone who has never heard of a REPL before, it stands for ‘Read, Eval, Print, Loop’ and it is basically an interactive console where you can run your code. Lisps generally place a lot of importance on the REPL and Clojure is no exception. It’s one of the things that really makes Clojure great, instead of having to write unit tests or small programs and run them to see what a piece of code does, you can just evaluate it in the REPL, in real time. Check out Try Clojure to see it in action, follow the instructions at the bottom.

A perfect match

M and I sat together at my desk and I loaded up a sample of incoming data as a Clojure map, keywords to values. The keywords had been generated by the system providing the data so M knew exactly what they meant. I then showed him the current mapping on the left and called the mapping function in the REPL to produce output for that data. M could clearly see where the data had come from and quickly pointed out that the wrong date was being used. A quick change and synchronize the REPL and we tried again.

This time it was a bit less clear, M could see that something was not right but wasn’t sure which fields were available, the incoming data was a big nested map with vectors of nested maps so it was difficult to distinguish what was what. A bit of ->> with some map and filter and a sprinkling of keys made things clear. The REPL had enabled us to zoom in on data and hide what was unimportant and clouding our vision.


Our session lasted about 15 minutes and at the end the mapping was 100% correct, I had live confirmation from a user. The code went live and into production and everything now works correctly. It was clear; our short session pairing at the REPL had saved days if not weeks of back and forth, getting things slightly wrong and reading through spreadsheets. It was a lesson that has saved me many more painful hours since and I am sure in the future.

Conclusion

I have not seriously used another language that would have enabled anything like this responsiveness in this particular situation and it has shown me how important it is to use the best programming language for the job. In this case, that language was Clojure.