Recently, as part of one of our client projects, we used a combination of multiple Azure Functions to invoke various API’s for enhancing the client’s incoming streaming data. The data flow was such that the Azure functions picked up and pushed the enhanced data through event hubs prior to and after processing the data which was ultimately stored in a table in a SQL Database. This activity was only an aspect of the architecture, the full pipeline was more complex but the above was the most interesting part and makes a good read so will be the focus of this post.
Azure Functions are units of code or “functions” in the cloud, providing serverless compute. They can be written to just solve the problem at hand without having to worry about the infrastructure to run them. A bonus for developers is that they have a choice of the following languages to write the Azure Functions – C#, F#, Node.js, Java, Python or PHP. They allow the developer to set the incoming source and outgoing source through the concept of input and output bindings.
The general syntax for an entry point of an Azure Functions is as follows:
But what if you needs to make the function ‘async’ to handle multiple threads and processes? This was exactly something what was done for this project and with it come some complexities.
Changing the method from ‘void’ to ‘async Task’ as such:
did not seem to suffice. Along with this came an issue; not being able to use the word ‘out’ for setting an output binding. Usually it is as easy as using the keyword ‘out’ (after the input binding) and then the output source type. The following compile time error will be encountered:
Frustrating, right? Well, there are a couple of solutions for this (the latter was used in the project). The easiest solution is:
1. Binding the output to the function’s return value.
You can then just return the value from function as illustrated below:
2. Bind the output to ICollector. Then add the item to the collector. This method is used when using more than one output binding.
The latter solution was used in the project to have a final entry point as such:
Note: Our own object type has been used as the input binding for the function. Azure Functions totally accept this, given that the incoming Event Hub MetaData maps exactly to the custom class properties – which is super helpful as no manual code conversion of mapping is needed.