Monday, January 12, 2015

Re-imagined Applications Demand Re-imagined Application Analytics

Traditional applications are being replaced with the many-to-many pairing of Apps to Services where core functionality is supplied via cloud-based software and delivered via a multitude of apps running across devices and runtimes. Beyond the obvious runtime combinatory complexity, the apps and services are typically developed by different organizations with independent release cycles under disparate business models. As a consequence, an application’s scope – the sum total of its software and content – has shifted from concrete to ethereal where ingredients can change or evolve from session to session.


Outdated analytics patterns can only offer limited insight

Analytics solutions built to focus on a single stack (or analyze stacks side-by-side), e.g. mobile apps or web sites or internal servers - or focus on a single stakeholder or persona, e.g. IT ops or web commerce – are poorly positioned to capture the dynamic, interoperable nature of modern application deployments or the increasingly diverse community of application stakeholders. 

Shared runtime data is the tie that binds components into an organic application

Apps track the services they call and services track the apps they serve through tokens and other shared parameters. Not every argument exchanged plays this role – consequently, a working knowledge of the components and their interfaces is required to effectively piece together individual sessions, users, and activities. 


PreEmptive Analytics: built for modern deployments and diverse stakeholder requirements

PreEmptive Analytics has been built from the ground up to offer an instrument-once and distribute-many approach supporting a portfolio of analytics endpoints as dynamic as the application components it monitors.

The following working sample illustrates how PreEmptive Analytics can instrument client and cloud components to provide unprecedented insight into app design, user behavior, and IT operations. The latest version of this app, the instrumentation, and the extensions to the PreEmptive Analytics Workbench can be found at GitHub PreEmptive Analytics Use Case Example

The sample app and the sample service


The sample app lets users submit anticipated expenses for pre-approval. The user identifies the expense category and estimated expense and submits the record to a managed service for centralized approval or rejection. The approval policies reside in the hosted service as do the historical records.

Every user session has an associated organization and unique user ID associated with it – this drives both policy and provides the “hook” to connect the client’s activity with the supporting software services.

The sample app is written in C#, instrumented with the PreEmptive Analytics API and then Xamarin is used to generate both Android and iOS instances.

The sample service is written in C#, also instrumented with PreEmptive Analytics and runs in an Azure Windows VM.

PLEASE NOTE – the analytics functionality demonstrated here is in no way dependent upon or specific to C#, .NET, Xamarin, Azure, Android, or iOS – this is one specific example to illustrate the general principles and capabilities of PreEmptive Analytics that can just as readily be applied to any WPF, Java, or C++ component – running on-premises and/or distributed across cloud services and devices.

Sample App Functionality


The app allows a user to time arbitrary workflows, throw exceptions, express preferences, and – last but not least – submit an anticipated expense for pre-approval.





















The sample managed service


A user selects the expense category and estimated expense and submits the information for approval.






















Based on the amount and other factors, the remote software service either approves or rejects the request. The client app informs the user in one of two ways.





As mentioned above, users can track arbitrary work flows that span (or work within) page and/or method boundaries by starting and stopping the following timer.






Identity

Also, as mentioned above – each user can observe the department and ID they are working under in each session.























PreEmptive Analytics Results


The following dashboards illustrate the cross-section of analytics supporting the full spectrum of application stakeholders from dev to DevOps to business owners.

Overview

The overview page requires no special configuration and is immediately populated (a latency measured in seconds) as runtime telemetry comes in from production. All versions of all components are available for inspection across client devices and cloud-services.

Even the vanilla overview page offers insights across component and stakeholder domains as illustrated by these four "feature" stats - all relate to the "expense approval" activity - but each represents a different perspective - providing insights into all of the moving pieces that come together to create the integrated user experience.




























Timing may be everything but all time is relative

User behavior, user experience, application service level, managed service service levels

Even without any special configuration, PreEmptive Analytics automatically breaks out usage and timing of:
  1. The Azure-based approval service (item 1 above) – IT operations cares about this perspective, 
  2. The client-side call up to the Azure-based approval service (item 2 above) – dev and devops cares about this perspective, 
  3. The time spent on the “mobile page” for expense approval (item 3 above) – UX design/dev care about this perspective, and 
  4. The time inside the larger workflow that leads a user to the mobile page (item 4 above) – app owner cares about this perspective

The close up of the feature tracking panel shows that 688ms of the client request is outside of the time actually consumed by the Azure service itself (690ms – 2ms). It also shows that once a user lands on the expense page, they spend almost 40 seconds filling it out and lastly, that the true workflow that takes the user into and out of this page is just over 50 seconds on average.























Application service levels


Deeper analysis is readily available as well – here the max, min, and average times that clients need to fulfill a client request are shown over time – alongside a “threshold” indicating a service level goal for the client-side service.












Business activity


PreEmptive Analytics combines the multi-tiered instrumentation outlined above with application-specific data capture and analysis – enabling powerful business activity insights. The following chart shows the volume, ratios, and trending of expense request approvals versus rejections over time. This particular instrumentation is generated from the cloud-based service – ensuring an enterprise-wide view across applications, platforms, and users.

Server record of expense requests













Segmentation


PreEmptive Analytics goes far beyond counting occurrences of Application-specific data – any data point can be used to segment runtime telemetry as well – providing powerful, contextual insights as illustrated below.

Usage and experience


Recall that each client session is assigned a department (or role) and a user ID. The following panel breaks out usage, users and exceptions by organization (a server-side lookup of the user ID) AND by the role.














NOTE that these dimensions can also be used as filters allowing stakeholders to focus on the most important organizations and the most importance roles inside those organizations. Below is a view into approvals and rejections by organization and role.

Business activity








Selecting any combination of organizations and roles sets the focus to the most important constituents to my operation – for the first time, I can segment, monitor, and optimize for the organizations and people that matter most. 


Bias by organization

A user can simply select an organization (which is indexed through a CRM look-up of the license key as the data streams in from production at runtime) and usage, stability, and quality of only users from that organization (one or more can be selected).






After selecting "Up And Away Inc." you can view both system activity and a business activity summary.







Bias by role

Similarly, selecting just the "VIP" role shows VIP activity across organizations.  



Keep in mind that the data in these tables is a "joined" view combining client-side information (role and activity) and cloud-based computing (request approval statistics).







The same business optimization can be applied to production incidents to support DevOps and support. The following panel shows the activity by user ID drilling down into specific exceptions.

Bias for DevOps



































What’s next?

If your business is (or will soon be) dependent on applications whose logic is distributed across devices and runtimes and you believe that application development should be AT LEAST as customer-centric and attuned to your business’ priorities as any other part of your organization – then upgrading application analytics needs to be a priority (not much different than building application security into the dev process and not after).

Contact me directly or visit PreEmptive’s site to explore how we’re helping organizations develop their application analytics practice to improve quality, satisfaction, and development ROI.