This blogpost is about a script…
Azure Service map
Azure service map is a really cool thing. It basically shows you all network connections your servers make when running. You can view this information in a table or in a visual form.
Now, does this look interesting? Sure it does, I see the sever, the connections it makes to other servers, I see the processes that start the connections… all of that in an easy to use interface. When migrating to cloud this is really valuable information. When doing firewall projects or management also. It helps you decide if you should open that port or not. There are more cases where this information is a huge time saver.
I am not going to write on how to setup this, the microsoft documentation is clear and up to date. Read all about it here: https://docs.microsoft.com/en-us/azure/azure-monitor/insights/service-map. You need 2 agents installed. Alternatively, if you already run SCOM, you don’t need to install the agents, you can link up SCOM to Log Analytics
TLDR: you install the agents agent on your Windows or Linux sever, you create a log analytics workspace in Azure, you connect both… et voila.
While I am a huge fan of Log Analytics, this blogpost is about using jArchi to draw relations. As a regular user of Archi you probably have created drawings where you linked servers & applications to each other to form something that supports a business proces. Service maps would definitely be a valuable source of information for that, because it shows you what application processes (application components) on what server (node) are in place.
Once you have the plugin, you can work with it using the information provided here: https://github.com/archimatetool/archi-scripting-plugin/wiki
This blogpost is about a script, a script for jArchi..
The script will use log analytics data to draw an Archi model with nodes and relations. it will show you information on the relation as well.
You can find the script here: https://gist.github.com/gevaertw/5048676da755dccfabe1947050b24fc8
How dos this work?
So, how can you use it?
The starting point is that you have a running setup of Azure service map, sending data to a log analytics workspace.
We need to get the data out of log analytics so that we can import it into the model.
- Open the query editor
- Enter the query
- Run the query
- Export the data, make sure to select only displayed columns
The log analytics query is included in the jArchi script, meaning; if I ever publish a new version of the script, an matching log analytics script is included. You find the log analytics script in the script code like this:
------Start of log analytics query----------------------------------- VMConnection | where Direction == "outbound" | where DestinationIp != "127.0.0.1" | distinct Computer, ProcessName, SourceIp, DestinationIp, Protocol, DestinationPort, RemoteDnsCanonicalNames, RemoteCountry) -------End of log analytics query------------------------------------
- Lots of RemoteDnsCanonicalNames are not filled in. This is because service map could not resolve them. Still.. not a problem, we have the remote IP, the script will create a node based on the remote IP instead of the name
- Lots of RemoteDnsCanonicalNames point to different hosts in the same network. In my case there were 50+ different CDN hosts, followed by a similar number of AWS hosts. None of the names made any sense at all..
Now, as said before this script uses jArchi. Make sure you have it installed in Archi..
Now, depending on the number of lines you have in your CSV this might take a while. On my average laptop a CSV with 1900 lines takes 90 seconds or so.
The script adds some properties, it indicates that the relation or node was created by the script, it also adds a timestamp. In case of the relation it also adds the origination process. In a later version I might use an application component, but for now let’s keep it simple.
Our 1900 lines of log analytics data generated an Archi model of around 1 MB. Models this “big” might no perform well anymore. You can reduce the number of lines by:
Some traffic is known and default. Your monitoring servers, SCCM, AD… You know them, so filter that traffic out using the where clauses. You can also filter out protocols like DNS, the same here, you know it is there, it brings no value
With heavily connected systems you will find al lot of unknown IP addresses on the internet. Depending on your needs the specific address is not interesting. group them all into a node called “internet” or so. The same for connections to pubic cloud providers and CDNs
Removing the process, or port
Depending on your needs the source process is interesting or not. If you don’t need it, remove it from the file. The same for processes and ports.
Excel is great for that, and more. When you cleaned your file, use excel to remove all duplicate records. Keep in mind that Excel might change the delimiter based on your settings. The script expects a comma as delimiter.
Wrap up & Conclusion
Hopefully you can use this script to help you build your inventory. You should however not see this as a replacement for service map itself, it lacks the performance and feature wise I cannot beat Microsoft (and I have no ambition nor desire to do that). It is meant als the bridge between Log Analytics and Archi, that’s no more, no less. Feel free to drop me a comment. I might add some extra features later on, based on what I need in my projects, or your cool suggestions. I cannot give support however…