Analytics Systems
While many analytics tools have analysis capabilities built in, often there’s a need for various dedicated analytics processes that can apply complex algorithms and generate advanced metrics without relying on client-side tooling to do so.
Analytics and BI systems in many ways can be likened to a traditional 3-tier application stack. Let’s consider a standard web application: in that structure, the three tiers are Presentation (the web site itself), Domain\Business Logic (underlying business rules and the “brains” of the application) and the Data layer that deals with storage and data integrity.
Similarly, analytics systems can often be realized the same, with the Presentation layer comprised of visualization and reporting tools like Power BI, the Data layer equatable to warehouses, lakes and data marts, and the “brains” or Domain\Business Logic being similar to the Analytical piece that sits between.
The Analysis portion of an analytic stack isn’t always required as a true entity; that is, many base level tools incorporate this into their service offerings and there may not be a need to realize this as a dedicated entity in a simple analytics solution.
However, when complexity grows and especially when analytical metrics become reusable across various points, it can be worth dedicating some effort into this area.
Essentially, the analysis portion of an Analytics\BI system is the one that does the serious number crunching and heavy lifting in terms of generating hard measures and metrics. Many data platforms have the capability of performing reasonably complex queries to generate metrics, and visualization tools typically provide a robust set of tooling to do the same. In many cases, these are sufficient to generate the measures for providing the insights your organization may need. However, the data platforms’ primary focus is on managing the storage and processing of incoming data, and the visualization platforms’ primary focus is for displaying data.
Analysis-specific tooling such as SQL Server Analysis Services provide a dedicated service highly optimized for generating, storing, caching, and re-using complex, resource intensive metrics that need to be generated and re-used across many visualizations or departments. In addition, analysis services tend to be an excellent source of organization and documentation material for KPIs and complex metrics.
- Analytics servers provide a means to generate complex analytics utilizing the power of underlying server or cluster resources and delivering the generated measures to the visualization client.
- These often work from data in data marts (or directly from warehouses) and can further isolate specific sets of data to visualization tools (e.g., a given metric might be calculated slightly differently depending on the target audience or use case; analytics servers can coordinate this as baseline for various cubes)
- Often, the data model of an underlying warehouse or mart is not optimal for a given analytical algorithm; analysis servers can be used to further cross-correlate and shape clean warehoused data into something optimal for algorithmic processing while still utilizing the resources of a backend system
- Allows for advanced data caching such as in-memory data sets that help with performance
- Essentially, analytics services are all about preparing structured warehouse data for presentation
- Optional for many systems (and often can be accomplished either via direct warehouse queries or from the visualization client (e.g., power bi)), but can really bring a data system to the next level when the scale is warranted