Panorama is a project about making Azure environment observable, giving Azure users insights to deloyed resources, performance, and security. Certain KQLs are adapted from Azure Monitor Community.
The workbooks does not currently contain Location parameter, as workbooks and dashboard are built for the purpose of a local Goverment Commercial Cloud program where all resources are in Southeast Asia region.
What Panorama offers?
Artifacts - Workbooks & Dasboard
How to setup a Workbook
How to setup a Dashboard
Collect Logs
Guide - Azure Network Monitoring Tools
Panorama contains supplementry monitoring viewpoints in the form of Workbooks and Dashboard in addition to existing community and built-in Azure workbooks. The workbooks are built on requirements gathered from a local goverment commercial cloud program and some differentiating info these workbooks provide are:
- What NSG rules are being created or updated, who did it and when?
- What Azure Firewall rules are being created or updated categorized by NAT, Network and App rules, who did it and when?
- What are the Threats Azure Firewall has protected me against? IP of the source of threats and when?
- A summary of all writes and deletes happening in my environment, who did it and when
- What are the network traffics within a VNet(Intra-VNet), inter-peered VNets, to and from between any External Public IPs or Azure-owned control-plane Public IPs?
- For each of my VM, how much disk space does each logical drive are left with?
- How much memory available currently out of total VM memory?
- Which VMs have missing Critical and Security Patches and what are the missing patches?
Panorama consists of the following artifacts:
- Workbooks
- Dashboard
- Copy Json content of any Panorama Workbook for example IaaSInsights,
- In Azure Portal, go to Monitor and click on Workbook, select Empty Workbook
- Follow by clicking Code icon "</>"
- Under Gallery Template, delete existing Json and paste in IaaSInsights Workbook Json content and hit "Apply"
- Save Inventory Dashboard as .json file on your machine.
- Upload .json dashboard.
- all general resource writes and deletes of your Azure environment
- Network Security Group changes, who made the change, what is the exact rule and the value changed. E.g: Access change from Deny to Allow, Destination Address to */Any.
- Azure Firewall rule changes, who made the change, a breakdown on exactly which NAT, Network and/or Application Rule is changed and the vault of change
This workbook contains 5 tabs:
- VM Availability: A simple graph view of Azure Health metrics. Green means VM is available, Orange can means stopped or not available for whatever reasons.
- CPU & Memory: This gives you the details of each VM what is the current CPU consupmtion in % and the number of cores. For memory, it shows the Available Memory out of Total Memory of the VM spec.
- Disk Capacity: Breaks down all drives attached to each VM and shows the free space in each drive. (D drive is a temp drive). Any disk lesser 15GB of free space will be marked Red.
- Patch Status: This tab gives you a Grid Summary and Detail view where you can select a VM in the Summary Table while the Details Table below shows you which patches are missing.
- Change Tracking: Showcase 5 areas of changes: File, Windows Services, Linux Daemons, Software and Registry
This workbook categorizes traffics that are processed by Azure Firewall into 4 tabs filtered views of: Threat Intelligence, NAT, Network and Application FQDNs
It operates 1 workspace at a time, contains a view of ingested logs in GB grouped by Monitoring Solution, and Kubernetes specific logs collection by enabling ContainerInsignts.
Traffic Search allows you to filter by source and destination VMs, flow types like Intra-VNet, Inter-Peered-VNets, Azure-owned PIPs and external PIPs.
This gives you valuable network traffic insights to your environment.
The data is based on "NetworkMonitoring" and "AzureNetworkAnalytics_CL" Log Analytics tables created when you enable Network Performance Monitoring and Traffic Analytics. See Collect Log.
It contains 3 general categories:
- All Resources
- Virtual Machines: Number of Linux and Windows VMs and which VMs are running and which are not. Pie charts of VM grouped by Image Type and by VM SKU.
- Networking: All Virtual Networks and Subnets
-
Networking - NSG: whistleblowing which NSG has no rules,
Subnets with no NSG (excluding AzureFirewallSubnet and GatewaySubnet)
and which Inbound and/or Outbound rules has destination address * | Any | 0.0.0.0/0 -
Networking - Public IP: Shows all resources with Public IPs and Public IPs not assiciated to any resource.
-
This section provides a summary of the available Azure network monitoring tools and how to setup and use them.
Do check out a couple of great videos on entire Azure Monitor landscape and another that covers all current Azure networking tools.
- Azure Load Balancer Insights
- Network Performance Monitor (NPM)
- Azure Monitor - Networks Insights
- Traffic Analytics (NSG Flow logs)
What this does?
- Topology visuals of load balancer IP -> LB rule to backend-pool resources with colored link-lines to show traffic health
- Availability: Backend pool health probe by backend VM IP addressa nd by Port. E.g: Backend VM can have healthy probes on port:80 but not port:443 due to no SSL cert setup.
- Data Throughput: How much Megabytes is going through your Load Balancer by Frontend IP and Port
- Flow Distribution:
- Inbound: Total count of request/traffic coming into LB and distributed to each of the backend VM
- Outbound: Total count of outbound request/traffic created by eech of your VM
References: video
What this does?
Focusing on 2 aspects of NPM (exclude ExpressRoute monitoring):
Performance Monitoring and Service Connectivity Monitoring (SCM).
In a nutshell, NPM provides visual topology and info on VM to VM packet loss and latency(Performance Monitoring),
and Http-based SaaS/PaaS URL monitoring(SCM).
-
Performance Monitoring
MMA agents in each VM acting like a "health beacon" send health probe(synthetic transactions) packets via TCP/ICMP to each other several times to measure and capture info like latency and packet percentage loss. Few terms to figure out before one can effectively use NPM solution UI:- Nodes: Actual VMs in subnets auto discovered from any VM with healthy MMA agent sending Heartbeat to workspace.
- SUBNETWORKS: Actual subnets of a VNet. Subnetworks are auto discovered when Nodes are discovered, therefore Subnetworks can't be manually added unlike NETWORKS.
- NETWORKS - This is similar to a VNet but its a logical network container for all discovered Subnetworks and can be any friendly name with spaces. Although Network could be a 1:1 to a VNet but it gives flexibility to group subnets in a another logical way apart from actual VNet.
-
Service Connectivity Monitoring
The same MMA agent in this case triggers connectivity to test Http, Https, TCP endpoints similiar to Network Watcher - Connection Troubleshoot.
The use cases for SCM therefore are broad:- TCP test - SQL Server, MySQL, FTP, SSH, SFTP servers
- Http/s test - VNet-Injectable-PaaS like API Management Premium, Integrated Service Environment, Azure Function(Http-triggered) running in Azure Kubernetes and more
- Https test - Internet SaaS endpoints
References:
This feature gives you an overall plus drill-down view on all nerworking resources in your environment.
Below screenshot with navigation guiding on each feature.
-
Networks Insights - Load Balancer Dependency View
This feature is the same as Load Balancer Insights -
Networks Insights - Application Gateway Dependency View
Key Metrics explained
(Backend connection time, Backend first and last byte response time, total time, client round trip time)Click on Dependency View icon to go to Application Gateway Metrics
References:
- What is Traffic Analytics
- Traffics Analytics Table AzureNetworkAnalytics_CL Schema
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent